verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
< kris__>
lozhnikov: Do you have the script that you used for generating the images and from the rbm with deep learning.net example. Also the cpp file of the test. I seem to have it missing.
< lozhnikov>
kris__: I wrote that for the ssRBM but it isn't difficult to use the binary RBM instead.
< lozhnikov>
Could you describe your build issue more detailed?
< kris__>
The build issue had to do with the cstdin not found since i think pywrapper were introduced that require that library.
< kris__>
I was able to build after removing the line from CmakeLists.txt that attaches -stdc++=libc++ when the apple mac os version is less that 10.9. I remove the if statement and it worked.
< lozhnikov>
which branch are you compiling?
< kris__>
The newest branch. Since then i have reverted the changes to the merge i did with latest version. now i run 2.2.4 everything works fine now.
< lozhnikov>
I haven't got MacOS. So, probably it is difficult to reproduce
< kris__>
Okay no problem. Right now i got working using the easy solution. Will look at a detailed solution later.
< kris__>
Both the resize layer and cross entropy with logit are succesfully build. Travis is faililng on randomForestTest.
< kris__>
So you can go ahead and have a look at them.
< lozhnikov>
I'll look through that today
< kris__>
ok, i will give the orilley example a final try today....
< lozhnikov>
kris__: the gradient of the generator is zero
< lozhnikov>
I tried to replace GaussianInitialization gaussian(0, 1); by GaussianInitialization gaussian(0, 0.02);. Actually, the oreilly example uses 0.02 as the standard deviation
< lozhnikov>
I got the same result
< lozhnikov>
Curiously, SigmoidLayer + CrossEntropyError works fine with it
< lozhnikov>
so, the error that we observed yesterday happened due to incorrect deviation
kris1 has quit [Quit: kris1]
< lozhnikov>
however, CrossEntropyErrorLogits works with deviation 1.0
kris1 has joined #mlpack
kris1 has quit [Client Quit]
kris1 has joined #mlpack
< kris__>
Just saw the comments...
< kris__>
So does the sigmoid + cross entropy layer work.....
< lozhnikov>
sigmoid + cross entropy shows the same results as CrossEntropyErrorLogits with deviation 0.02
< lozhnikov>
but the gradient of the generator is zero
< kris__>
Okay i will check what wrong now.
< kris__>
btw i have updated the summary of the work. Will you have a look at it when you have time.
< lozhnikov>
actually, that isn't the summary
< lozhnikov>
you should write the final blog post and describe all changes you have done
< kris__>
I do not know why it doesn't show up on the website though.
< lozhnikov>
try to change "Summary Date" by "Date"
< lozhnikov>
1. "I am happy to say that in terms of visual reconstruction in both the examples of Mnist digit generation and Gaussian distribution generation we were able to get comparable results with keras and tensorflow."
< lozhnikov>
I think we didn't get good results on the mnist dataset yet.
< lozhnikov>
2. "Here are the results of our implementation on the digits dataset(smaller version Mnist)."
< lozhnikov>
Actually, this is the mnist dataset
< kris__>
yes mnist 7 i would update it. Okay i will try to run it for more number of epochs and see if we can get better results.
< lozhnikov>
3. "One of the reasons accuracy of ssRBM is less than ssRBM".
< lozhnikov>
I guess "is less than binary RBM" is correct
kris1 has quit [Quit: kris1]
< lozhnikov>
4. "I would also like to say here that Mikhail tried to convert mu-ssRBM code for testing our implementation but it took a lot of time finally."
< lozhnikov>
I finished the implementation of the mu-ssRBM. But it isn't well tested yet, I focused on the GAN PR. That's why I haven't pushed the code yet. So, I think there is no sense to write about that. However, it's up to you.
< lozhnikov>
5. "We tried ssRBM on the cifar data set code but due to the large volume of data set and scarcity of the computation resources, we decided that it was not really required."
< lozhnikov>
If I remember right, last time you told that you got good accuracy on the CIFAR dataset.
< zoq>
kris__: About the libc++ issue, what OS did you use?
< zoq>
kris__: Also the static site generator is really picky about the metadata header, if you remove the extra lines (between the meta data) the build should pass
< lozhnikov>
kris__: 6. I think it is reasonable to upload the tests to github gists and mention them in the blog post.
< lozhnikov>
Except the comments above the blog post looks good to me
< lozhnikov>
kris__: I looked through the oreilly example again. The fourth ReLU layer is not needed in the generator network
klitzy has joined #mlpack
klitzy has quit [Client Quit]
< kris__>
I will have to look at the cifar classification accuracy
< kris__>
I have the code for it
< kris__>
But I Have to look at the results once
< kris__>
Zoq I use Mac OS X 10.12 if I rember right
< lozhnikov>
kris__: without the fourth ReLU layer the gradient of the generator network isn't equal to zero
< kris__>
No i was training on the CrossEntropy with Logits.
< kris__>
since the training was going on since morning i din't kill it.
< kris__>
I will kill it now.
< lozhnikov>
how much time should the test take?
< kris__>
I am not sure. I actually did not run the orilley example since you said that you had run it already.
< lozhnikov>
Mini-batch SGD provides an output each epoch. So, you can estimate the time
< kris__>
Okay i will check and tell you but i am pretty sure that one epoch takes > 25 min on my machine with m = 2000.
< kris__>
lozhnikov I had a question you said that mnist 7 digit reconstruction we got was not good. I think that could be because of the regularisation of weights.
< lozhnikov>
kris__: and what is the question?
< kris__>
No i was just saying that we don't have regularisation of weights in mlpack so i was trying with higher number of epoch to get a images what were smooth.
< kris__>
But i am unable to do that.
< lozhnikov>
maybe you are right. I didn't dig into the keras example
< lozhnikov>
kris__: have you fixed the blog post and have you submitted the final evaluation? I just want to make sure that you haven't got troubles with that
< kris__>
Not yet. I will do that by tonight. I just wanted to wait for the results from the cnn gan but i think that would not be possible i would go ahead.
< kris__>
Should i submit it now. Or wait till tuesday. Because if you submit once you can't change it.
< lozhnikov>
I don't recommend to wait Tuesday
< lozhnikov>
so, I think it is better to complete the evaluation now
< kris__>
Ok i will do that in hour or so then.
vpal has joined #mlpack
vpal has quit [Read error: Connection reset by peer]
vivekp has quit [Ping timeout: 248 seconds]
vivekp has joined #mlpack
vivekp has quit [Ping timeout: 260 seconds]
vivekp has joined #mlpack
< kris__>
zoq: The images are not displayed in the mlpack.org page.
< kris__>
but on github page they are shown do i need to give full path for img src right now i have given a relative path only.
vivekp has quit [Read error: Connection reset by peer]
< lozhnikov>
try to fix the path i.e. replace "../images/mnist_out.png" by "images/mnist_out.png"
< kris__>
lozhnikov: Should i implement the batch norm layer for the cnn test. Also do you have any ideas how we could test it faster.
< kris__>
Also one more thing was batch normalization acts as a regularizer it can be replaced by the DropOut layer. But when i was using the dropout layer the network converged very quickly i mean in 2-3 iterations. This is true for even individual ffn's.
< kris__>
zoq: Why do you say the present implmentation of batchNorma won't work with conv layers.
< kris__>
*batchNorm
vivekp has quit [Ping timeout: 276 seconds]
< zoq>
kris__: The current implementation does not cover the data dimensions correctly if > 2 (normalization step isn't correct); Sumedh is working on that end.
< zoq>
kris__: Looks like you fixed the image issue?
< zoq>
kris__: I'm not able to reproduce the libc++ issue, Mac OS > 9 (Mavericks) should automatically link against libc++, did you say that the problem encountered with mlpack 2.2.5?
< kris1>
Yes the newest version.
< zoq>
Did you build with -DBUILD_PYTHON_BINDINGS=OFF?
< kris1>
no actually i did not…..
< zoq>
Do you have time to test that?
< zoq>
not sure that's the problem
< kris1>
Well that would fix it i think
< kris1>
I will check in the morning tommrow. I have a early day tommrow.
< zoq>
okay, thanks a lot
vivekp has joined #mlpack
vivekp has quit [Ping timeout: 255 seconds]
vivekp has joined #mlpack
kris1 has quit [Quit: kris1]
kris__ has quit [Quit: Connection closed for inactivity]