10:57 < zoq> kris1: Hello, about the sharing weights question, couldn't you implement the visible and hidden layer as a single layer, that way you could use the same weight internally? If that's not an option, you could write another layer that holds the visible and hidden layer and handles the weight sharing. I guess if you know your architecture (static) you could also, implement weight sharing on a higher level.

15:22 < sumedhghaisas_> zoq: Hey Marcus

15:22 < sumedhghaisas_> free?

15:22 < sumedhghaisas_> you are right about the github comment...

15:22 < sumedhghaisas_> I was thinking something better...

15:23 < sumedhghaisas_> why don;t we shift to list

15:23 < sumedhghaisas_> I noticed that both backward and forward access elements in order

15:23 < sumedhghaisas_> so an iterator would prodocue similar results

15:23 < sumedhghaisas_> what do you think?

16:04 < rcurtin> sumedhghaisas_: sorry I haven't chimed in on that discussion yet, I'm currently really underwater with work (some things exploded here) so it might be tonight before I can get to it

16:31 < zoq> would be relative easy to test, don't you tink?

17:08 < sumedhghaisas_> zoq: already doing it :)

17:08 < sumedhghaisas_> and also... I was doing some debugging

17:08 < sumedhghaisas_> I realized that we are calling Evaluate many times before Gradient

17:08 < sumedhghaisas_> with same parameters

17:09 < sumedhghaisas_> even though with tests the overhead is not too much...

17:09 < sumedhghaisas_> but it might be huge for large datasets

17:27 < kris1> YEs

17:28 < lozhnikov> kris1: you told yesterday that visible layer requires an access to hidden biases

17:29 < kris1> Yes

17:29 < kris1> it does

17:29 < lozhnikov> If I am not mistaken the hidden layer and the visible one use different biases

17:29 < lozhnikov> but the weight matrix is shared

17:31 < kris1> Not the weight matrix but hidden biases. For both layer’s the weight and not weigthts matrix are shared

17:34 < lozhnikov> I suggest to use another notation:

17:34 < lozhnikov> hidden variables are sampled from P(h=1|v)=sigm(Wv+c)

17:34 < lozhnikov> and visible variables are sampled from P(v=1|h)=sigm(W^t h+b)

17:35 < lozhnikov> So, the matrix W is shared between the visible and hidden layers

17:35 < kris1> Yes the c is for the hidden biases

17:35 < kris1> and not the visible biases

17:35 < lozhnikov> and biases c and b are different

17:36 < kris1> W is shared but the weigthts indicate all the parameters of the networks and weight indicates W

17:37 < lozhnikov> okay, in such a way the matrix W is shared and biases 'b' and 'c' are not. Do you agree?

17:37 < kris1> Yes

17:39 < lozhnikov> I suggest to use the same technique for sharing this matrix as the technique that allows to store different wights inside the main weight matrix

17:43 < kris1> Sorry i do not understant you

17:45 < kris1> I am not sharing the weights explicitly every layer has there own parameters(weights(weight + bias)) while we update ‘weight W matrix in the vanilla_rbm_impl.hpp ’ them we update them with same values

17:45 < lozhnikov> Just use the following constructor: arma::mat(weights.memptr() + offset, nRows, nCols, false, false); with the same offset

17:45 < kris1> Though i wanted to share weights between layers but i have no idea how to do so

17:46 < kris1> There are 2 layers visible and hidden there are instantiated sepreatlely and fed to the vanilla_rbm_impl.hpp

17:46 < kris1> The weights can’t be shared in the above way you specify

17:47 < lozhnikov> Why? You can write something like visibleLayer.Parameters()=arma::mat(hiddenLayer.Parameters(), ... ... ... )

17:50 < kris1> oh okay so you are suggesting that in vanilla_rbm_impl while in the constructor we do visibleLayer.Parameters() = arma::mat(hiddenLayer.Parameters(),….,….) this would share the full weights(weight + biases) both the network

17:50 < kris1> sorry for both the layer i mean

17:51 < kris1> that would do the trick its just that we would have to cautious while we reset the parameters

17:51 < lozhnikov> yeah, you allocate the full matrix and then just set up links

17:53 < kris1> Okay that would do the trick i guess. I will ask if i get anyother doubts

17:54 < lozhnikov> ok

18:24 < zoq> sumedhghais: Are you sure the parameter are the same? Can you share the code/test you used for debugging?

18:26 < zoq> sgupta: Do you have a book with awesome sayings? ... nice blog post :)

18:27 < sgupta> zoq: thanks a lot. And no, I don't have a book. It's just something I shine at. :)

18:28 < zoq> sgupta: really unique style :)

18:34 < sgupta> zoq: thanks a lot :)

18:37 < zoq> shikhar: Excited to see the next step: do you plan to implement HOGWILD in one week?

18:40 < shikhar> zoq: Probably not :) I have some things to discuss about the paper right now. I have an implementation at hand, but the code is a bit complex.

18:41 < zoq> shikhar: I can imagine, it's not going to be easy :)

18:42 < shikhar> The implementation is from the authors of the paper. I wonder why they couldn't use OpenMP :D

18:46 < zoq> shikhar: Maybe you will find out, but hopefully not :)

18:50 < zoq> sgupta: It somehow reminds me of Ryan's comment style he used for the IRC frontend/backend; it's not the same style, but it also makes small things way more entertaining.

18:50 < zoq> Here are some examples:

18:50 < zoq> https://github.com/rcurtin/irclog/blob/master/scripts/gen-nav-html.sh

18:50 < zoq> https://github.com/rcurtin/irclog/blob/master/scripts/process-log.sh

18:50 < sgupta> zoq: sure I'll look at them :)

18:55 < zoq> I mean the comment about the sed command isn't helpful at all, but I'm not sure this was his intention anyway.

19:05 < rcurtin> I forgot that I committed those comments... :)

19:06 < rcurtin> irssi log format has not changed so we are okay for now :)

19:06 < sgupta> zoq: yes indeed :)

22:37 < rcurtin> ironstark: did you resolve any of the Shogun installation issues, or are you still having issues with that?