07:52 < partobs-mdp> zoq: I've more or less done with adding separator. Let me share some observations:

07:53 < partobs-mdp> zoq: (more of a question) In my opinion, we should not add optional parameter addSeparator to AddTask - it's already a part of task specification

07:53 < partobs-mdp> zoq: In CopyTask I had worse results when I added the separator (on maxLen = 6 model quality dropped form 26% to 14%)

07:59 < partobs-mdp> zoq: (once again, more of a question) In that comment (https://github.com/mlpack/mlpack/pull/1005#issuecomment-313075884), what did you mean by the arma::mat methods (fixed length) - could you elaborate on that?

08:21 < partobs-mdp> zoq: By the way, I'm re-reading the code and I can't really see a place where separator can help (except CopyTask, but so far it proves to be detrimental). What am I missing?

10:21 < zoq> partobs-mdp: About fixed length, right now we use arma::field to store a vector of different length. In my experiments, I've tested a fixed length representation for each sequence (max length over all sequences) and stored the results in arma::mat. So I think it's a good idea to provide an interface that takes arma::mat as input instead of arma::field.

10:21 < zoq> partobs-mdp: I agree adding an end marker for the Add task, has no real value.

14:56 < partobs-mdp> zoq: Almost made it (refactoring + adding static-length sequence generator). However, I'm badly stuck with some strange error which forces g++ to produces 27k+ lines of error messages (hard luck). It occurred after my changes in AddTask, which should have wrapped it up. The current (buggy) code is in my last commit to the task PR.

15:09 < zoq> partobs-mdp: let's see

15:22 < zoq> partobs-mdp: I think, the default parameter causes the errors you see, see my latest comment. Have to build the code to take a look at the error messages.

15:25 < partobs-mdp> zoq: Moved default parameters, still nothing. I'll attach the `make` output in a moment

15:27 < partobs-mdp> zoq: https://gist.github.com/partobs-mdp/e24d96b07979111c0f8b9a10ac68d348

15:44 < zoq> partobs-mdp: Looks like it's related with the HAM unit code, can you remove that code from the PR?

15:55 < zoq> partobs-mdp: hm, it's not, removed all HAM releated code

21:36 < sumedhghaisas> zoq: hey marcus. The gru layer tests are complete. Although some other tests are failing. Don't know why.

21:38 < zoq> sumedhghais: I see the FastMKS serialization test is failing.

21:39 < sumedhghaisas> zoq: yeah. what should we do about that?

21:39 < sumedhghaisas> also had a question about NTM memory implementation.

21:39 < sumedhghaisas> I should initialize the memory matrix with some random initial value right?

21:40 < zoq> sumedhghais: As long as it's not related with the code changes, we don't have to worry about this, but if you like to take a look feel free :)

21:41 < zoq> sumedhghais: Yes, I think I've seen someone using an LSTM network to for the intialization, I can see if I can find the paper.

21:42 < zoq> But, I don't think that's something we have to do.

21:44 < sumedhghaisas> zoq: I was thinking about a linear layer to which we pass a value. The linear layer can be passed through some non-linearity and then reshaped to get a memory matrix. This way the layer can be trained as well to decide the initial values.

21:44 < sumedhghaisas> what do you think?

21:45 < zoq> Sounds good, I guess a linear layer is fast enough to get some reasonable results without spending too much time on the initialization.

21:47 < sumedhghaisas> yeah. Thats what I was thinking. And a tanh non-linearity. Not a very complex structure.

21:49 < zoq> Yeah, if we make it somewhat customizable we can test different strategies, what do you think? Since initalization is important, that might make sense.

21:50 < sumedhghaisas> I agree. I will try to structure it such that its easily customizable. I am adding a memory layer in ANN layers so that we can use it in RNN framework.

21:51 < sumedhghaisas> Do you think thats a good idea?

21:52 < zoq> yeah, absolutely

21:54 < sumedhghaisas> Okay. I will try to templatize that thing to accept memory initialization policy.

21:56 < zoq> Okay, nice, I'll look over the PR tomorrow, looks good so far.

22:16 < sumedhghaisas> zoq: Hey marcus, do we have a reshape layer?

22:17 < zoq> sumedhghais: not yet :)

22:18 < sumedhghaisas> ahh... Okay I will need that for NTM :) won't be that hard...

22:18 < sumedhghaisas> although I am not sure about the API

22:18 < sumedhghaisas> do you have any ideas?

22:23 < zoq> hm, actually do we need a reshape layer? Couldn't we take the output of linear and use memptr() or something like that, if we create the memory of the NTM?

22:29 < sumedhghaisas> zoq: thats true. But if we do the same thing with abstraction, it might easier to read the code. But then memptr() can't be used I guess.

22:35 < zoq> hm, true, I guess a simple layer would take the the output size as parameter:

22:35 < zoq> class Reshape

22:35 < zoq> {

22:35 < zoq> Reshape(output size)

22:35 < zoq> Forward(....)

22:35 < zoq> }

23:04 < sumedhghaisas> zoq: okay I will think about that. Do you know any good strategy to differentiate the circular convolution?