10:33 < ashishtilokani> Now mlpack_test is running.. all I needed to do was copy the mlpack_test.exe from bin to the build folder where the test data was stored and ./mlpack_test from there. But If want to just tinker around with a method and see if it is still working correctly what is the approach?

10:40 < zoq> ashishtilok: Hello, we are using the Boost Unit Test Framework to write tests. Every test contains a so called "TestSuite" (BOOST_AUTO_TEST_SUITE(TestSuite)). you can run only the tests in that test suite with 'bin/mlpack_test -t TestSuite'. A specific test case called 'TestCase' (BOOST_AUTO_TEST_CASE(TestCase)) could be run with 'bin/mlpack_test -t TestSuite/TestCase'. I hope this will be helpful.

11:01 < ashishtilokani> for example, if I have to use load_save_test.cpp, then what command do i have to use exacty? I think 'TestCase' needs to be replaced with something in bin/mlpack_test -t TestSuite/TestCase

11:07 < zoq> The name of the TestSuit can be found in line 16 "BOOST_AUTO_TEST_SUITE(LoadSaveTest);". So if you want to run all tests in that TestSuit just run 'bin/mlpack_test -t LoadSaveTest'. If you are just interested in the results of the e.g. HDF5 test case run 'bin/mlpack_test -t LoadSaveTest/LoadHDF5Test'. For more test cases take a look into the 'load_save_test.cpp' file.

11:35 < ashishtilokani> ./mlpack_test log_level=test_suite --run_test=LoadSaveTest This works..

11:39 < ashishtilokani> now if I change the cpp files to be tested and add some more tests in the test file... What all is needed to be build again?

13:59 < billLiu> Hi, everyone! First, Merry Christmas!!! And I'm a beginner to use mlpack. Yes, it's absolutely a cool toolkit and easy to use. The main purpose I use it is to train GMM in speaker recognition. Until now, it works.

14:00 < billLiu> But I wonder whether things would get even better. I found that when I use GMM::Estimate, it just use single core(top show 100% cpu usage) in my server which has 32 cores. It's definitely a waste and takes times longer than my lab's implementation which is commercial and close source.

14:00 < billLiu> So I wonder whether it is possible to utilize multi-core to train GMM using mlpack? Anyone who can help? Thx so much!

14:02 < billLiu> BTW, I have manually compile Armadillo using ICC and MKL, and mlpack itself is compiled using ICC. The command I used is like "icpc train_gmm.cpp -o train_gmm -O2 -fp-model strict -fomit-frame-pointer -xhost -larmadillo -lmlpack -lboost_program_options -Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group"

14:44 < naywhayare> billLiu: merry christmas to you too :)

14:44 < naywhayare> you are right that GMM training is single-core

14:45 < naywhayare> there are a couple of easy options -- the first is to use OpenBLAS which is parallelized, and the second is to use OpenMP to parallelize the training of GMMs

14:46 < naywhayare> anyway, I personally don't have time right now to rewrite the GMM code with OpenMP but if you wanted to I would gladly accept a patch :)

15:14 < billLiu> hey, naywhayare. The second one is what I'm thinking about and I will add openMP once I check it works right. And the first one... actually I use intel MKL library while is multi-thread, to compile Armadillo, Does it means the process would be parallelized if I do that?

15:16 < billLiu> Or I should do something to compile mlpack?

15:17 < naywhayare> billLiu: if MKL is parallelized, then there's no need to use OpenBLAS

15:17 < naywhayare> and if it is parallelized, then any parts of mlpack algorithms that call Armadillo methods that use the MKL will also be parallelized

15:18 < naywhayare> but I'm not sure how much of the running time of GMM training is spent in those calls

15:18 < billLiu> yes, i know

15:19 < naywhayare> I have to run for now, but I'll be back maybe later today and if not I'm basically always in the channel and will see messages that you leave :)

15:19 < naywhayare> you can also use the mailing list or the bug tracker to get in touch if you need

15:19 < billLiu> I will read the corresponding code to see if it is necessary to use openMP

15:19 < naywhayare> talk to you later :)

15:19 < billLiu> OK, thx !!!

15:19 < billLiu> talk you later :)

15:55 < naywhayare> ashishtilokani: to rebuild after you change a file, you just need to 'make mlpack_test' and then run the tests you like :)

16:00 < ashishtilokani> Thnx... I will try it and see if it works.

16:33 < ashishtilokani> There are two copies of load_impl.hpp. One in the build directory and one in the folder before the first time I used make on mlpack. Where do I need to change the code to see changes?

16:36 < ashishtilokani> I guess the file not in the build directory. The one in the build will get updated after using make I guess.

16:38 < ashishtilokani> One thing which bothered me is that just for a small change in one file, the mlpack_test is build again which takes a LOT of time. Is there anyway we can quickly test somethinga single file.

16:53 < naywhayare> ashishtilokani: unfortunately the answer here is no, because lots of the code in the mlpack test files depends on load_impl.hpp

16:53 < naywhayare> if you have a multicore system, you might try a parallel make, i.e. 'make -j4 mlpack_test' (if you had four cores, for instance)

16:55 < ashishtilokani> When I changed load_impl.hpp , the changes were not reflected after I used make mlpack_test.

16:56 < naywhayare> did you change it in build/src/ or src/?

16:56 < ashishtilokani> Any file I modify, to test it, only mlpack_test needs to be build?

16:56 < ashishtilokani> src/

16:57 < ashishtilokani> build/src/ gets updated once make mlpack_test is executed I guess

16:57 < naywhayare> right

16:57 < naywhayare> you could try 'make clean' and then 'make mlpack_test' and see if that resolves the issue

16:57 < naywhayare> it might take a little while though :(

16:58 < ashishtilokani> :(

16:59 < ashishtilokani> I really think testing of load_impl.hpp can be done in a quicker manner but don't know how.

17:01 < naywhayare> it is possible, it's just tedious

17:01 < naywhayare> if you really wanted, you could modify the CMakeLists.txt in mlpack/src/tests/ and comment out every file except load_save_test.cpp

17:01 < naywhayare> but you would need to remember that if you wanted to test anything else, you would need to uncomment it

17:01 < ashishtilokani> that would work for me

17:07 < ashishtilokani> The changes were not being reflected because I had copied mlpack_test.exe from build/bin/ to build/ as all the test data was in this directory.

17:08 < ashishtilokani> After using make, the new mlpack_test was in the build/bin and I was still using the old one in the build directory.

17:15 < naywhayare> ah, yeah; maybe easier to just run 'bin/mlpack_test' :)

17:26 < ashishtilokani> Yeah, that would work. No need to copy it again and again

18:53 < ashishtilokani> I tried inplace_trans() on a 10k * 676 by adding it as test in load_save_test and the process got KILLED

18:54 < ashishtilokani> the file size was 228.9 MB

18:56 < ashishtilokani> I guess inplace_trans() should not be used as they are damn slow for large matrices and not required for smaller matrices.

18:58 < ashishtilokani> You might want to close #209 as wontfix

18:58 < ashishtilokani> or something like that

19:01 < ashishtilokani> or Is there anything I should do regarding #209

19:01 < ashishtilokani> ?

19:35 < zoq> hmm, that's weird, I've tried 'arma::mat A = arma::randu<arma::mat>(10000, 1000); arma::inplace_trans(A);' and everything is works just fine.

19:35 < ashishtilokani> how much time did it take?

19:41 < zoq> 0.744542s

19:42 < ashishtilokani> I tried again, this time it executed pretty quickly... I wonder why the first time my computer become unresponsive

19:44 < ashishtilokani> The previous time I used trans() then printed the entire matrix on command line and then used inplace_trans() in a single test case

19:45 < ashishtilokani> this time i just used inplace_trans()

19:45 < zoq> printing such a matrix should be time consuming

19:47 < ashishtilokani> you are right but it appeared as if the prinitng had stopped

19:47 < ashishtilokani> one thing i did not understand was how to print the total time taken

19:47 < ashishtilokani> in a task

19:48 < ashishtilokani> Timer::Start("intrans"); Timer::Stop("intrans");

19:48 < ashishtilokani> Log::Warn<<Timer::Get("intrans"); does not work

19:50 < zoq> right, because Timer::Get(...) returns a timeval struct

19:50 < zoq> timeval t = Timer::Get("intrans");

19:50 < zoq> Log::Debug << t.tv_sec << "." << std::setw(6) << std::setfill('0')

19:50 < zoq> << t.tv_usec << "s";

19:51 < zoq> this should work

19:51 < ashishtilokani> ok

19:52 < ashishtilokani> can you suggest me some non trivial bugs i should try?

19:53 < ashishtilokani> or any new algorithm I can implement which can be added to mlpack

20:03 < zoq> hm, I think most of the tickets are non trivial, some a really hard to fix because you need to know exactly how the method/algorithm works.

20:03 < zoq> If you like to implement a new algorithm it is up to you to pick something you find interesting. If you interested in trees I guess ryan really likes to see some new trees (e.g. octree). But there is so much you can do, I can't just recommend something.

20:06 < ashishtilokani> thnx... I would try to find something interesting and also read about octree