verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
navdeep has quit [Ping timeout: 252 seconds]
ImQ009 has joined #mlpack
travis-ci has joined #mlpack
< travis-ci>
manish7294/mlpack#76 (impBounds - 9edda26 : Manish): The build has errored.
< zoq>
ShikharJ: Great results, as you said in the blog post, we can merge the code in the next days.
< zoq>
ShikharJ: I'm curious, what's the meaning of the last word?
< Atharva>
zoq: I left a message here yesterday about the sequential layer, can you please check
< ShikharJ>
zoq: I'll get to optimizing the code and the additional SSRBM test in another PR for now.
< zoq>
ShikharJ: Great!
< zoq>
Atharva: Let's see.
< ShikharJ>
zoq: Actually, I end my blog posts with a farewell message in different languages, that is a Sanskrit word for "see you again".
< zoq>
ShikharJ: Neat :)
< zoq>
Atharva: hm, what layer did you use before the seq layer?
< Atharva>
the reparametrization layer
< Atharva>
but I don't think it matters what layer we use before it. If the first layer in the seq layer is any layer that does not have the outputWidth() function, then if the second layer if convolutional, the inputWidth and inputHeight gets set to 0
< zoq>
Atharva: I see, I'll create simple example to reproduce the issue.
< Atharva>
Okay
< zoq>
The issue I see is, that right now we do a single Forward step to get the output width/height for the next layer.
< zoq>
But for the seq layer there is no single forward step, since it will call the Forward function for all layer in a row.
< zoq>
But this would only effect a cascad of layer that implement the width/height function; but you said your case looks like embedded -> Seq (Linear -> Conv) -> ...
< zoq>
so, the first conv layer should receive the correct width/height
< Atharva>
Yes, so what do you suggest?
< zoq>
Not sure yet.
< zoq>
Does the seq layer hold multiple conv layer in your case?
< Atharva>
Yes it does.
< zoq>
okay
< Atharva>
I don't think that matters, even a single convolutional layer after the linear layer creates the same problem
< zoq>
So for now, we could we could comment out reset and manually set the width/height.
< zoq>
Does that work for you?
< Atharva>
zoq: Okay, so that means we don't hard set it to true
< zoq>
Right
< Atharva>
Yes, it does. That's what I am doing locally to get it running
< zoq>
but that should only work if width/height doesn't change
< Atharva>
Sorry I didn't get you completely, width/height doesn't change with what?
< zoq>
Ignore the last point, it does work.
< Atharva>
Okay then, I will make a commit to my latest PR
< zoq>
Actually, is there any reason to force a reset at all..
< Atharva>
Yeah, I couldn't understand why it was hardcoded to true as well
< zoq>
Atharva: For your vae model did you test another optimizer e.g. Adam?
< Atharva>
hmm, I tired sgd with Adam update
< Atharva>
Why exactly? Will it be better if I try different optimizers?
< Atharva>
tried*
< zoq>
ahh, right I was talking about AdamMax or RMSProp
< zoq>
Adam is just an alias for SGD<AdamUpdate>, so yes it's the same
< Atharva>
No, I haven't tried anything else than Adam. Should I try some other Optimizer for the next model I will be training on binary MNIST?
< zoq>
if you can start an experiment sure, I can also test it out
< zoq>
just wanted to ask first
< Atharva>
I can, there are some more models I and Sumedh have planned to experiment with. Is there any specific reason to try out different optimizers? What aspect of the models do you think they can affect?
< zoq>
Generally escape from a poor local minima.
< Atharva>
Oh!
< Atharva>
I will try RMSProp and AdamMax for the basic vae model then and see if the loss goes down further.
< zoq>
Adam should work for the VAE model, but I think it's easy to test it.
< Atharva>
Sumedh did seem to think that it should be lower than ~120
< Atharva>
yeah
< Atharva>
BTW, do you have any idea why the ReconstructionLoss with a normal distribution isn't working on normal MNIST, the code that I mailed you
< zoq>
Atharva: As soon as I found something I'll let you know.
< Atharva>
zoq: Sure, sorry about the frequent reminding