ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
< davida>
Hi. I am trying to apply GradientClipping on a plain Stchastic Gradient Descent. I though I should use VeanillaUpdate but GradientClipping does not seem to like that. Here is the code snippet and the error.
< davida>
Never mind. Think I am using it wrongly.
< davida>
Nope. Cannot get it to work. Seems like GradientClipping doesn't like VanillaUpdate
< davida>
Any idea what I am doing incorrectly?
< davida>
The error is:
< davida>
class GradientClipping
< davida>
class GradientClipping
< davida>
Hmm. Can't seem to paste the error in here.
< davida>
gradient_clipping.hpp:40:3: note: no known conversion for argument 3 from ‘mlpack::optimization::VanillaUpdate()’ to ‘mlpack::optimization::VanillaUpdate&’
< davida>
Hah. Sorry guys. Should learn to read the errors better. I had an anomolous () on the declaration of my VanillaUpdate.
vivekp has quit [Read error: Connection reset by peer]
vivekp has joined #mlpack
< davida>
zoq: Is there a reason that we need to put a layer before adding a recurrent layer in the RNN model? I am referring to the example in the recurrent_network_test.cpp
< davida>
... where the first layer is IdentityLayer
< davida>
zoq: Also, was the process to have different sample lengths in an RNN incorporated with the latest release and how to use it?
davida has quit [Ping timeout: 252 seconds]
davida has joined #mlpack
< zoq>
davida: Per default we discard the backward step for the first layer, since it's not going to be used. But that doesn't work if the first layer holds other layers, so we just add a dummy layer. There is an idea to only skip the layer if it's doesn't implement the Model() function.
< davida>
OK - so adding the IdentityLayer basically does no modification to the input but allows the discard.
< zoq>
right
< davida>
zoq: Any update on the different sized input sequences for the RNN? If you recall changing Rho each training step caused some errors.
< zoq>
davida: Ahh, right, thanks for the reminder, have to put it on the list
< davida>
... but to complete the RNN exercise I need a way to stop the training else it learns that 0s (I pad with zeros) are most likely to be the next element.
< davida>
Which is not the case
< zoq>
I see, hopefully I can get this done this week
< davida>
Thx. Will be a great help for a real world problem I have to manage as well.
< zoq>
absolutely
< davida>
BTW - if you implement it for RNN will that mean it will also work for the LSTM model?