10:35 < zoq> ironstark: Hello, have you had a chance to write a blog post yet? let us know if there are any issue.

11:40 < shikhar> zoq: Are builds on Jenkins allowed to use more than 1 core?

11:47 < zoq> shikhar: Depending on the job, yes if we start e.g. the matrix build bascially all 72 cores are used, since we start a bunch of jobs in parallel, so we have to find some reasonable values.

12:02 < shikhar> Alright, thanks :)

12:02 < shikhar> The Travis job finished but the times are nearly the same

12:04 < zoq> okay, I guess in this case no need to go with -j4

14:02 < rcurtin> there's not really good support for load balancing with Jenkins, so I'd suggest to use only one core for each build on Jenkins

16:03 < sgupta> rcurtin: hi! I am taking a debian as base. My question is, since we are creating a new user let say mlpack who doesn't have root privileges, doesn't we have to build mlpack beforehand inside the container?

16:34 < sgupta> rcurtin: I have tried lot many things and for debian, the size is 387 MB in which ~270 MB is our dependencies and rest is base image and necessary packages.

19:09 < zoq> ironstark: Pelican is picky when it comes to the metadata, 'Date: 2017-06-07 15:00:00' should work.

19:12 < ironstark> zoq: Updated

19:13 < zoq> ironstark: Looks great, thanks for the update.

19:14 < ironstark> :-)

19:14 < zoq> Actually I think if someone is interested to extract the meta information from the commit itself, I'm here to help :)

20:12 < rcurtin> sgupta: I'm not sure what you mean, the idea of the container is to provide an environment in which mlpack can be built, so we can't build mlpack in the Dockerfile

20:12 < rcurtin> do you want to open a PR with the smaller debian image? I can take a look and see if I can think of any more optimizations

20:58 < kris1> zoq: While running sgd we update the iterate('parameters') using the currenct_function('data point') at each iteration. we update the parameters of network. But now the parameters of the network have only changed right but the parameters of the layers that comprise the network have not changed. can you explain this? I need this for writing the cd-k algorithm

21:06 < zoq> kris1: hm, I think I have to see the code, to help you with the issue

21:07 < kris1> which code do you mean the sgd.hpp??

21:08 < kris1> or my cd-k code ??

21:09 < zoq> The code that you use to produce the issue.

21:11 < zoq> Something with a simple test case (something I could run), would be great.

21:12 < kris1> Well here is the cd-k code https://gist.github.com/kris-singh/233090e8febb92371243aeb12e585f6a.

21:13 < kris1> I have not written the rbm model so writing test would be a little difficult.

21:13 < kris1> Here is the revised vanilla rbm implementation https://gist.github.com/kris-singh/a5de37f17d68c9d11fbdb05bcb57dafc

21:16 < kris1> in the case of rbm i would want to forward propagate after the parameters of the network have been changed after finding the gradients on one of the data points. But for running the forward function, I would have to call the forward_visitor on all the layers in the network.

21:16 < kris1> My question is that how did the weights of layers that make up the network got updated

21:17 < kris1> Did you get my point?

21:17 < zoq> ah, ah, I think I misunderstood your question, you don't have an issue, you like to know how the layer parameter are updated, right?

21:19 < kris1> Yes

21:20 < zoq> So, that's relative simple, you call the optimizer with your parameter matrix:

21:20 < zoq> optimizer.Optimize(parameter);

21:20 < zoq> The parameter matrix contains all layer parameter. You basically pass the layer parameter to the optimizer.

21:21 < zoq> So, if you have one layer e.g. Linear you basically do: optimizer.Optimize(linear.Parameters());

21:22 < lozhnikov> https://www.irccloud.com/pastebin/yFgwfEbI/

21:22 < zoq> If you have more than one layer you can use the FFN class, or another class, that does the wrapping for you.

21:22 < lozhnikov> kris1: If I understand your question right that happens here (core/optimizers/sgd/sgd_impl.hpp:111 for example)

21:22 < lozhnikov> updatePolicy.Update(iterate, stepSize, gradient);

21:23 < zoq> right

21:24 < zoq> In your code it's: iterate += stepSize * gradient;

21:24 < zoq> btw. template<typename RBMType>CDK<RBMType>::Optimise(arma::mat iterate) should be template<typename RBMType>CDK<RBMType>::Optimise(arma::mat& iterate)

21:26 < lozhnikov> kris1: I'll look through your code in detail tomorrow and then I'll add some comments (now it's too late, it's time to sleep:)).

21:27 < kris1> okay lozhnikov:

21:28 < zoq> Btw. has anybody seen a good movie lately or can recommend something?

21:29 < rcurtin> I saw a movie called "CHAPPiE" some weeks back, I very much enjoyed it

21:29 < rcurtin> it's by the same director who did District 9

21:29 < rcurtin> but I dunno if that is the kind of movie you are looking for :)

21:29 < zoq> that's the one with the robot right?

21:30 < zoq> I guess, I have seen a poster.

21:32 < kris1> zoq: parameters matrix is a local matrix right . I do get the point that we update the parameter matrix. But every individual layer also has its own parameters. which we set using boost::apply_visitor(WeightSetVisitor(std::move(parameter),offset), network[i]); here we are setting the layer parameters for every layer in the network. But we do not do this when we are optimise.

21:33 < kris1> Does the parameters matrix contain reference's(address) to the parameters of the individual layer matrix. I don't think so>

21:33 < kris1> ?

21:35 < zoq> Sounds interesting, speaking of District 9, waiting for part 2

21:35 < zoq> The other way around, the layer bascially hold references to the local parameter matrix of e.g. FFN

21:35 < lozhnikov> kris1: look at methods/ann/ffn_impl:128

21:36 < lozhnikov> The FFN class passes parameters to the optimizer

21:37 < rcurtin> zoq: yeah, it is an interesting scifi story based around AI

21:38 < rcurtin> like many movies the technical details can be shaky at times but at one point they do actually correctly run some code on GPUs

21:38 < kris1> Yes, i think i get it now. The individual layer parameters are reference's to the parameter matrix of the FFN network. That would make sense

21:38 < rcurtin> weird to see a monitor in a movie with a valid bash shell running valid commands :)

21:39 < zoq> :)

21:39 < zoq> Have you seen Ex Machina? really liked the setting.

21:39 < zoq> kris1: yes, right

21:40 < kris1> So basically WeightSetVisitor makes the layer parameters point to the network parameters am i thinking correctly

21:40 < lozhnikov> kris1: yeah

21:41 < kris1> Ahhh great!!! thanks lozhnikov zoq

21:44 < kris1> lozhnikov i just wanted to ask you one thing i have implemented the visible and hidden layer right now. But they don't much right now ie their forward function serves as an identity function. Should i change that and sigmoid(Linear(input) + bias) but that would require me to template the visible layer to accept Linear and sigmoid layer.

21:44 < kris1> or should i just make a visible layer and concat it with linear and sigmoid layer typedef that layer

21:46 < rcurtin> zoq: I haven't, I'll add that one to my list

21:46 < lozhnikov> kris1: I think you can define Linear and Sigmoid as private members of the visible and layers

21:47 < lozhnikov> *and hidden

21:47 < rcurtin> I've been wanting to see Shaft for a few weeks now but that is a very different type of movie :)

21:48 < kris1> Hmmm but that would restrict it's usage.....template would be a better solution i guess

21:48 < kris1> lozhnikov:

21:49 < lozhnikov> I think it is definitely a good idea to pass VisibleLayerType and HiddenLayerType to the RBMType class as template parameters

21:50 < zoq> rcurtin: Not sure I heard of Shaft, just looked at the wiki page.

21:50 < lozhnikov> kris1: So the implementation of these layers will completely define the type of the RBM

21:52 < kris1> Yes i am doing that now. But i am implementing the forward_visible function in the rbm right now using forward_visitor. I actually wanted to move it to the visible layer but that would require me to either use linear and sigmoid as primitive types or as template parameters to the visible layer

21:53 < kris1> template parameters would give it more flexibility in case someone would later can also use linear+tanh for defining the visible layer

21:55 < lozhnikov> kris1: ok, I understand you. Maybe that is a good idea

22:01 < zoq> rcurtin: Btw. did you get a chance to look into: https://github.com/mlpack/mlpack/issues/1011#issuecomment-304718538

22:06 < zoq> I can fix the issues, but I think in case of e.g.

22:06 < zoq> adjustedScore = lastScore + lastQueryDescDist;

22:06 < zoq> adjustedScore = lastScore + lastRefDescDist;

22:06 < zoq> you know what to do

22:10 < rcurtin> yeah, I have some changes in progress

22:10 < rcurtin> I'll push what I have to a branch later tonight

22:10 < rcurtin> the score issue needed some further thought

22:12 < zoq> ah, okay great