12:29 < nilay> zoq: i will have to implement 1 x 1 convolution separately, right?

12:30 < nilay> because we need to accumulate depth here unlike 3 x 3 or 5 x 5 convolutions (for which we can use NaiveConvolution)

12:39 < nilay> and also then backprop would be different for 1x1.

12:40 < zoq> yes, it's probably a good idea to implement 1x1 separately

12:42 < nilay> yeah so we should implement 1x1 layer right?

12:42 < nilay> separate forward and backward pass

12:43 < zoq> You can also implement the 1x1 conv inside the inception layer

12:44 < nilay> but then if we want to use just 1x1 conv separately, then we would not be able to

12:46 < zoq> Good point, I guess, you are right a 1x1 conv layer is the way to go.

12:46 < nilay> ok thanks

12:50 < nilay> zoq: do you get what is being said here: https://www.facebook.com/yann.lecun/posts/10152820758292143

12:51 < nilay> how fully connected layers are just 1x1 convolution

12:56 < nilay> i'll be back in 10 minutes.

12:58 < zoq> You can express a fully-connected layer using a 1x1 convolution, Right now, I can't remember what paper proofs this. I think it was connected with Yann's binary conv network.

13:00 < zoq> But, I'm not sure they are the same.

13:00 < nilay> ok can we do the reverse?

13:01 < nilay> express 1x1 convolution as a fully-connected layer

13:04 < zoq> https://www.quora.com/Is-a-fully-connected-neural-network-conceptually-similar-to-a-1x1-convolutional-neural-network

13:04 < zoq> you mean instead of implementing a 1x1 conv layer we could just use the fully-connected layer?

13:05 < nilay> yes, i can think how we could do the forward pass

13:05 < nilay> i am thinking about the backward pass

13:07 < nilay> ok it would take much more time, its better to implement it as separate

13:08 < zoq> nilay: Hold on, why do you think we can't use the ConvLayer to do 1x1 convolution?

13:09 < nilay> no, because i couldn't find a method that does (arma::cube input, arma::mat filter, arma::mat output)

13:10 < nilay> sorry can't use NaiveConvolution

13:10 < zoq> I think we could just use the ConvLayer inside the inception layer.

13:16 < nilay> how

13:16 < nilay> can't even do the convolution operation

13:19 < zoq> ConvLayer<> convLayer0(1, 8, 1, 1);

13:19 < zoq> convLayer0.Forward(input, output)

13:19 < zoq> does a 1x1 convolution on the input data.

13:20 < zoq> ConvLayer<> convLayer0(10, 2, 1, 1); is probably is a better example, because you like to reduce the input data

13:24 < zoq> The last example takes an cube with 10 slices as input and outputs the results of 2 (1x1) convolutions.

13:27 < zoq> so let's say the input is arma::cube(100, 100, 10) the output would be of size arma::cube(100, 100, 2)

17:41 < rcurtin> mentekid: are you done with the multiprobe LSH PR? I think it is ready to merge, but I dunno if you were waiting to do anything else

17:42 < mentekid> No I think it's ready

17:43 < mentekid> I've started working on parallel find() and unique() code but things don't look good... especially find(), there's too much lock contention

17:43 < mentekid> so it ends up being way slower tha arma::find()

20:07 < travis-ci> mlpack/mlpack#1135 (master - a50784d : Ryan Curtin): The build was fixed.

20:07 < travis-ci> Change view : https://github.com/mlpack/mlpack/compare/812048c7c6be...a50784d84b26

20:07 < travis-ci> Build details : https://travis-ci.org/mlpack/mlpack/builds/141433633

23:51 < zoq> Karl_: Just realized you were talking about kpca, if you get the kernel right, It's probably a good idea to test the nystroem method.