ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
< zoq>
rcurtin: Looks like the last git commit build timed out, but I can't increase the build time, some random Jenkins error; maybe it works for you?
qur70 has joined #mlpack
qur70 has quit [K-Lined]
< rcurtin>
jeffin143[m]: yep, next release will be 4.x. I think there are a few PRs linked to mlpack 4.x that can be finished and merged now too
< rcurtin>
I guess, at some point someone (usually it is me but it doesn't have to be :)) should think about the minimum we need for a 4.x release
< rcurtin>
now we are done removing the linked boost dependencies, but the other boost header dependencies would be harder to remove so probably that is infeasible for 4.0.0 at least
< rcurtin>
zoq: strange, I got a random Jenkins error too
< zoq>
rcurtin: Maybe a restart helps, I guess easy to test?
< rcurtin>
I have our workspace backed up (as of about a week ago, at least), so let's see what a restart does
< rcurtin>
hopefully it won't nuke the entire configuration
jenkins-mlpack2 has quit []
jenkins-mlpack2 has joined #mlpack
< rcurtin>
same thing...
< rcurtin>
I'll try upgrading all the plugins and handling all the warnings on the "manage jenkins" page and see if that makes things better
jenkins-mlpack2 has quit [Remote host closed the connection]
jenkins-mlpack2 has joined #mlpack
< NippunSharmaGitt>
Hi, I have a question. If I make some changes to the `ann` codebase then I am not able to find anything to just make the ann code again (something like `make mlpack_ann`). How can I do this ?
< NippunSharmaGitt>
Like there is `make mlpack_pca` for the `pca ` codebase
< NippunSharmaGitt>
Is there something I am ignoring that already exists ?
< zoq>
rcurtin: fingers crossed
< zoq>
NippunSharmaGitt: That only applies to executables (*_main), since there is no executable for the ann code there is no make ann
< zoq>
NippunSharmaGitt: But make will only build changes, and not the whole codebase again, if you are worried about that.
jenkins-mlpack2 has quit []
jenkins-mlpack2 has joined #mlpack
< rcurtin>
zoq: ok, I upgraded some stuff... but I ended up setting the timeout to 300 by manually modifying the config.xml file for the job
< rcurtin>
I think, if I wanted to debug further, I might consider deleting the job entirely and recreating it with the same configuration
< rcurtin>
I can't seem to find anything in the logs
< zoq>
Strange, I was able to modify the satic analysis job config.
< zoq>
I guess, no need to recreate, since it's working for now?
< rcurtin>
yeah, strange... if it's working for now, I wasn't going to mess with it further :-D
< NippunSharmaGitt>
@zoq So after I make changes to ann there is no need for me to make again (apart from the tests) ? Is this because all files in ann are .hpp or header files ?
< zoq>
NippunSharmaGitt: Not quite, you still ahve to call 'make' but if you run make again you will see that it will only build changed files, or files that include them
PriyankaK has joined #mlpack
PriyankaK has quit [Remote host closed the connection]
Priyanka has joined #mlpack
Priyanka has quit [Remote host closed the connection]
PriyankaK has joined #mlpack
< NippunSharmaGitt>
Okay thanks. Is there any way in which I can prevent making mlpack tests (because they take too long) ?
< zoq>
NippunSharmaGitt: If you want to disable all tests you can add -DBUILD_TESTS=OFF to the cmake command.
< NippunSharmaGitt>
(edited) ... thanks. Is there any way in which I can prevent making mlpack tests (because they take too long) ? => ... thanks.
PriyankaK has quit [Remote host closed the connection]
< NippunSharmaGitt>
@zoq Thanks for clearing
< NippunSharmaGitt>
One more question I have. So, just to check I added a member function to `ffn.hpp` named `getNum()` that returns a random `int` (I did that to just check that the changes are reflected in my main program). After that I did `make` (without the tests). But when in a separate .cpp file which I made to check the `getNum()` I always receive an error saying that `getNum()` is not a member function of FFN class.
< NippunSharmaGitt>
Is there anything I might be missing here ?
< zoq>
NippunSharmaGitt: THat that's like the correct approach, I guess what you can do is delete libmlpack.so just to make sure it was updated?
< zoq>
NippunSharmaGitt: Also the function is public right?
< NippunSharmaGitt>
I tried deleting libmlpack.so and then running `make` again but it still shows the same error.
< NippunSharmaGitt>
Yes, the function is public
< zoq>
NippunSharmaGitt: Maye there is another version of mlpack installed, or are you directly passing the patht to the generated lib?
extrowerk has joined #mlpack
< extrowerk>
Hi!
< zoq>
extrowerk: Hello there.
< extrowerk>
This is the season of mlpack-port-update, so lets do this.
< extrowerk>
How is there so many nicks? What happend?
< rcurtin>
extrowerk: we added bridges to matrix, gitter, and slack :-D
< extrowerk>
ah, i see
< extrowerk>
how doing nowadays? is there any big happening on mlpack front?
< NippunSharmaGitt>
@zoq I am manually passing path to lib folder
< rcurtin>
extrowerk: I would say so, we're refactoring to remove dependencies and make mlpack more useful for embedded applications
< rcurtin>
zoq worked hard getting C++ notebook environments working, so this makes a really nice use case for mlpack, where you can be a data scientist and do prototyping directly in a notebook, and then turn around and deploy that code exactly, since it's C++
< rcurtin>
much easier than the deployment process with python or other tools
< extrowerk>
nice
< extrowerk>
was mlpack always so small? it is just ~5 mb tarball.
< rcurtin>
it's only small until you compile it :-D
< extrowerk>
Also cmake seems to be auto-downloading some stuff.
< rcurtin>
you can disable that, if it makes your packaging job easier
< rcurtin>
we did split out our optimization toolkit into the "ensmallen" package, and CMake will automatically download that if it's not available on the system already
< extrowerk>
This is actually not ideal for us, a program should not download things during the compilation process, we want to build them in isolated environment
< extrowerk>
is it possible to build ensmallen separately?
< extrowerk>
if so i can create a recipe for that too.
< extrowerk>
also it seems it will be again a rabbit hole, i have to update hdf5 first
< rcurtin>
yes, I just mentioned, easy to disable; just specify -DDISABLE_DOWNLOADS=ON in the CMake configuration
< rcurtin>
then just make sure that ensmallen is available as a dependency, and also the STB image header files
< rcurtin>
(you can package mlpack without that, but it won't have image loading functionality)
< rcurtin>
for version 3.4.2, there's no longer a dependency on boost::program_options, but for that version there was still a dependency on boost::serialization and boost::unit_test_framework
< rcurtin>
the upcoming version 4.0 release (not sure when) removes those two dependencies too
< rcurtin>
(though boost headers will still be needed)
< extrowerk>
thanks for the hint, deps removed
< extrowerk>
i don't have to make a separate port for ensmallen/stb if i can download them and put them in the correct place manually
< shrit[m]>
@extrowerk ensmallen and stb are header only, doing make will build to tests, and then you can also use make to install them directly
< rcurtin>
extrowerk: it might be nice to make separate ports for those, since it's possible to use both STB and ensmallen outside of mlpack---but up to you
< rcurtin>
they should be much, much simpler packages than mlpack to package :)
< rcurtin>
like shrit[m] pointed out, they're just header-only C++ libraries with nothing to install but sources
< extrowerk>
i found the url for ensmallen, but it seems cmake download .h files for stb, is that right?
< rcurtin>
yeah, STB is not an mlpack-associated project, it is just some header files
< rcurtin>
yes, but you can disable the download and CMake will use stb_image.h found on the system if it exists
< rcurtin>
for Fedora I actually bundle stb_image.h and stb.h as part of the mlpack package, because it would be too much work to create a package just for two header files (lots of bureaucracy)
< extrowerk>
is this really the best way to do this?
< rcurtin>
I'm not sure what you're getting at
< extrowerk>
hunting .h files from different folders?
< rcurtin>
you don't have to use the automatic downloader
< rcurtin>
STB may or may not be packaged on a user's system
< rcurtin>
I'd really prefer if it was available in all package managers, but it's not
< rcurtin>
if it *isn't* found on the user's system, then we have an option to automatically download a version that we control where we know that it works
< extrowerk>
but mlpack depends on stb2.22 or on stb1.13? or which version?
< jeffin143[m]>
<rcurtin "you don't have to use the automa"> rcurtin (@freenode_rcurtin:matrix.org):
< rcurtin>
and yes, those two files are in different directories, because stb_image.h is versioned differently (!) than stb_image_write.h
< extrowerk>
ah
< jeffin143[m]>
Yes exactly that's what I was going to point out
< rcurtin>
sorry, I think I misunderstood the point you were making so I wrote a lot :-D
< jeffin143[m]>
Probably initially we planned to just
< jeffin143[m]>
Load images
< NippunSharmaGitt>
@zoq it seems to be working now. I had to remove previously installed mlpack from /usr/local/lib, /usr/local/include, /use/local/bin and then installed the one with code changes. Is there any other way that I can do this without installing mlpack ?
< jeffin143[m]>
And rather implemented Saving option too
< jeffin143[m]>
So we had to use write.h too
< jeffin143[m]>
> And rather implemented Saving option too
< jeffin143[m]>
Later*
< rcurtin>
jeffin143[m]: yeah, I think you are right about that. I don't like STB's "distribution model", where it's just a bunch of differently versioned header files in a repository
< rcurtin>
but, the functionality is really useful for us, so, we chose to go with it despite the difficulty of packaging STB
< jeffin143[m]>
rcurtin : True , it gave us nice features :)
< jeffin143[m]>
So it's ok to dodge it
< rcurtin>
yeah, we made sure because of this situation that you can compile and use mlpack *without* STB, for situations where it would be too difficult
< jeffin143[m]>
rcurtin (@freenode_rcurtin:matrix.org): time for new release may be 😉
< rcurtin>
perhaps, we need to figure out if there are any other "big" things to get in before 4.0 :)\
< rcurtin>
good thing jenkins doesn't run as root :-D
< zoq>
yep :D
< extrowerk>
i found the following notes in the Haiku mlpack recipe, i think i wrote it: "# mlpack easily eats up all your ram, so don't do parallel jobs!"
< zoq>
extrowerk: haha, that didn't change.
< extrowerk>
i noticed
_slack_mlpack_U7 is now known as ronakypatel[m]
< ronakypatel[m]>
Anyone tried saving the GRU model (data::Save in txt/bin format) on disk and then reloading (data::Load) to use for prediction (model.Predict)? I keep getting matrix multiplication error. I am able to save and retrieve model for prediction with LSTM/FastLSTM but not for GRU. GRU model works fine, if I use the model.Train and model.Predict in the same go (like the one given in RNN test to predict next
< ronakypatel[m]>
alphabet in Reber). Likely there is some small issue in saving or loading GRU model in text/bin format.
< shrit[m]>
When you try to save model, is the file empty? did you tried to see of there any thing inside?
< ronakypatel[m]>
This is the output model file, if it helps.
< shrit[m]>
in .txt
< ronakypatel[m]>
Sent you the file already!
< ronakypatel[m]>
I don't see any problems in the file. File has content and somewhat in the similar format as LSTM/FastLSTM
< shrit[m]>
When you have incompatible matrix, one of the matrices is empty
< shrit[m]>
even not initialized, so either there is an issue when loading the file, or there is nothing inside this file
< ronakypatel[m]>
The file has content as I said. So issue with loading the file.
< shrit[m]>
Can you print the content of the model? after it has been loaded?
< ronakypatel[m]>
Sure like model.Parameters()
< shrit[m]>
Of course
< shrit[m]>
Did you check the features? provided to Predict() function?
< ronakypatel[m]>
Do you mean the test data points.
< shrit[m]>
Yes
< ronakypatel[m]>
Yes I guess, and note that the prediction works if the model is build in the same go.
< ronakypatel[m]>
Train Model and use immediately for prediction
< ronakypatel[m]>
The feature set is same
< shrit[m]>
So where is the issue, you mean if you change the file?
< ronakypatel[m]>
Save the file on the disk, then load and then predict.
< ronakypatel[m]>
I need to use the model for future predictions
< shrit[m]>
I see, did you use the same model name used when you tried to load the file?
< ronakypatel[m]>
Yes, note LSTM/fastLSTM works fine.
< ronakypatel[m]>
Same code.
< ronakypatel[m]>
Just GRU
< rcurtin>
I would bet here that some member of GRU isn't being serialized in serialize(), but I'm not sure which one (currently underwater with some other things otherwise I'd dig in)
< ronakypatel[m]>
It's a beautiful implementation, with this minor issue.
< rcurtin>
ronakypatel[m]: if you're willing to get your hands dirty, you could try using GDB to catch the exception when it's being thrown, and see which matrices in the GRU class are being multiplied
< ronakypatel[m]>
Do you need to serialize to store in txt, bin format
< rcurtin>
ronakypatel[m]: right, each member of the GRU class that we need to persist needs to be serialized explicitly in the GRU::serialize() function
< ronakypatel[m]>
Digging deeper is the plan, if no help is there.
< ronakypatel[m]>
I mean if I can get help from the experienced people, I can use it. Otherwise will try to dig in.
< rcurtin>
ok, up to you :) I would love to help, but at least for now I don't have the spare cycles to dig into the issue more than just guessing :-D
< shrit[m]>
@rcurtin, also these data member are from 3.4.2,
< ronakypatel[m]>
I totally understand. Let me see if I can find from where the exception coming from. I will keep posted here.
< shrit[m]>
I give a look at GRU implementation
< rcurtin>
shrit[m]: true, maybe something has changed there since then since it now serializes via cereal, but I'm not sure
< rcurtin>
ronakypatel[m]: if you use gdb as a debugger, you can just type `catch throw` before you run the program and it will hit a breakpoint at the first thrown exception
< shrit[m]>
Yeah, I see, hopfully (if there is any bug) it did not passe the transition
< ronakypatel[m]>
I will try with the catch/throw, but it may take some time, the build process takes time.
< rcurtin>
ronakypatel[m]: yeah, it can take a little while---also be sure to compile your program with `-g` so it has debugging symbols :)
< shrit[m]>
@rcurtin there are some data members from the GRU classes which are intialized in LSTM,
< shrit[m]>
sorry
< shrit[m]>
I mean there are some data member are not serialized in GRU, that are intialized in LSTM,
< shrit[m]>
but I am not sure which of the data memeber need to be serialized
< ronakypatel[m]>
Here is the relevant stack Ryan Curtin , Let me know if you are looking for something different.
< rcurtin>
ronakypatel[m]: yeah, you'd want to take a look at stack 11 to see what two members of GRU were being multiplied, and figure out which one is empty
< rcurtin>
in this case it looks like the program was compiled without debugging information
< ronakypatel[m]>
Looks like I need to recompile entire mlpack with -g?
< ronakypatel[m]>
Not just my code.
< rcurtin>
I would think just your code would be fine, since basically everything in the ANN code is header-only, but perhaps not
< rcurtin>
if you need to recompile mlpack, and you already built it by hand, you can just use CMake to adjust the options:
< rcurtin>
$ cmake -DDEBUG=ON ../
< ronakypatel[m]>
I guessed so too
< rcurtin>
(from your build directory)
< rcurtin>
then just `make mlpack` (and that step should be pretty quick---building the tests is what takes a long time)
< ronakypatel[m]>
OK, lets see, will post the finding
< ronakypatel[m]>
Ryan Curtin I updated the stack file. I did not recompile whole MLPACK yet, but in previous debug session, some of the object files were carry over from previous build. I recompile the my program and the stack looks complete.