verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
vivekp has quit [Ping timeout: 264 seconds]
vivekp has joined #mlpack
prakhar_code[m] has quit [Ping timeout: 240 seconds]
prakhar_code[m] has joined #mlpack
Atharva has joined #mlpack
< Atharva>
About the parallelization of the ann module, what has been planned?
< zoq>
Atharva: We should implement EvalauteWithGradient for the FFN class as well, and we should run some benchmarks against OpenBLAS.
< zoq>
Shikhar already tested the GAN implementation with OpenBLAS but couldn't see huge performance improvements; perhaps there is some expression that needs to be rewritten.
< zoq>
Besides that, there is definitely potential to improve/parallelize the conv operation.
< ShikharJ>
zoq: I can implement EvaluateWithGradient function for FFN class for now. The thought hadn't occurred to me, this would be the fastest way to see a performance improvement.
< zoq>
ShikharJ: That would be great, but don't feel obligated.
< Atharva>
zoq: ShikharJ: Just to confirm, this is for multi-core cpu code execution, right?
< ShikharJ>
Atharva: Yes. But EvaluateWithGradient function is just for performance improvement of the optimizer update call.
< Atharva>
Okay, this seems really interesting.
< zoq>
Right, if you like to checkout the GPU performance an easy solution is to switch to NVBLAS, Bandicoot might work as well, but NVBLAS might be easier.
< Atharva>
zoq: Okay, I will check it out. Have you tried training models on mlpack using NVBLAS or Bandicoot?
< zoq>
Atharva: I used NVBLAS; for some models you get some pretty good speedups.
< Atharva>
zoq: That's great! What gpu did you use?
< zoq>
Atharva: GTX 1080 and GTX 960
< Atharva>
zoq: Okay, I just have a GTX 1050ti, but I think it will still give atleast some speedup.
< zoq>
Atharva: Yeah, if I remember right you need CUDA 6.0 support.
< Atharva>
zoq: Yeah, 1050ti is supported luckily. I did try training some models on tensorflow-gpu.
ImQ009 has joined #mlpack
< Atharva>
zoq: I was training a VAE model on MNIST. I have observed that my model is totally dependent on the step size. I mean if I use higher or lower than an optimum value, the loss saturates at a not so low value . I can understand that happening for a high learning rate, but why is it happening for a learning rate lower than the optimum one?
< zoq>
Atharva: What's the optimum? Two ideas, take a look at the initial weights, perhaps start with small values, the other idea is to check the gradients at the end, mayber some aren't correct.
< Atharva>
Manually training it on small models, I found out that at a value of 0.003 it gives very low error compared to step sizes higher or lower than that.
< Atharva>
How do I check if gradients are correct at the end of training?
< zoq>
Atharva: You could print the gradient at the end of the Gradient call either in the FFN class or in the optimizer class.
< Atharva>
zoq: Yeah I understand, but how do I check if they are correct? Sorry if this seems obvious to you, maybe I am forgetting something very basic.
< zoq>
Atharva: Ahh, good question, I would just check if they look "resonable", if one of them is MAXDOUBLE or a huge part is 0, that is a good indicator. From that point we could take a look at a specific layer.
cjlcarvalho has joined #mlpack
< Atharva>
zoq: Okay, thanks. I will try and see what's happening.
< zoq>
Atharva: Okay, let me know if you need any help.
caiojcarvalho has joined #mlpack
cjlcarvalho has quit [Ping timeout: 248 seconds]
< ShikharJ>
zoq: Are you there?
< zoq>
ShikharJ: yes
< ShikharJ>
zoq: I was wondering whether it would be better to merge the RBM PR as it is now, and provide the tests and batch support in subsequent PRs? I'm saying this because the current set of changes are huge (~1800 lines), and with the additional tests and BatchSupport, it might get increased even further. What do you think?
< ShikharJ>
zoq: It would take a lot of effort to review them all at once. Maybe we can initiate a review on the PR now, and I can, meanwhile, work on a couple of different branches regarding the additional features.
< zoq>
ShikharJ: I'm not sure the current code is correct, and I think a test would be helpful. We could split the batch support from the PR but I don't like the idea to split the tests as well.
< ShikharJ>
zoq: Hmm, I see. We should implement the tests first then.
cjlcarvalho has joined #mlpack
caiojcarvalho has quit [Ping timeout: 244 seconds]
< zoq>
rcurtin: If I remember right I had to explicitly set the trigger phrase in the job configuration, not sure you encounter the same issue.
jenkins-mlpack2 has quit [Ping timeout: 260 seconds]
< rcurtin>
it seems like it is triggering, but the Jenkins logs indicate there is a problem:
< rcurtin>
'Request doesn't contain a signature. Check that github has a secret that should be attached to the hook'
< rcurtin>
so I am digging into that now
jenkins-mlpack2 has joined #mlpack
< rcurtin>
ah, ok, I think I got it. I just had to ensure that the shared secret in the Github webhook setup and on Jenkins were the same
jenkins-mlpack2 has quit [Ping timeout: 240 seconds]
< zoq>
nice
jenkins-mlpack2 has joined #mlpack
< rcurtin>
anyone want to push a commit to an open PR?
< rcurtin>
I can't easily find the right kind of message to redeliver to Jenkins to see if it's working
< rcurtin>
too many comments recently, all the messages I could redeliver that I see aren't commits :)
< rcurtin>
ok, I think I have everything working mostly properly... I'll check again in the next couple of days
< rcurtin>
Atharva: sorry for the noise on #1441, it was the one I chose for testing Jenkins :)
witness_ has quit [Quit: Connection closed for inactivity]