11:10 < saksham189Gitter> Can you try implementing the two class classification only?

11:14 < sakshamb189[m]> Commonly used approaches for implementing SVM for multi-class classification are One vs. Rest and One vs. One which are both based on training multiple two-class classifiers. I think we could add support for that later on if required.

11:45 < HimanshuPathakGi> OK I will go for two class classification.

11:45 < HimanshuPathakGi> First

15:00 < KimSangYeon-DGU[> kartikdutt18 sakshamb189 Are you there?

15:01 < kartikdutt18[m]> Hey KimSangYeon-DGU , sakshamb189 , I'm here.

15:01 < sakshamb189[m]> Hey, I am here

15:01 < KimSangYeon-DGU[> Great, Ok

15:01 < KimSangYeon-DGU[> I think today's main topic is about batch normalization

15:02 < kartikdutt18[m]> Right.

15:03 < kartikdutt18[m]> The implementation of batch normalization as sakshamb189 pointed out is that it takes into input from a linear layer.

15:03 < KimSangYeon-DGU[> Yes

15:04 < KimSangYeon-DGU[> So, you guys mean we need to modify the implementation for the convolutional layer, right?

15:04 < sakshamb189[m]> No only the batchNorm layer

15:04 < kartikdutt18[m]> We need to add support for multiple features map / channels in batch norm layer.

15:04 < KimSangYeon-DGU[> Ahh, I mean batch normalization

15:05 < KimSangYeon-DGU[> * Ahh, I mean batch normalization for the convolutional layer

15:05 < kartikdutt18[m]> I have done the same for forward propogation (I haven't tested it but I think it's correct).

15:07 < kartikdutt18[m]> So the implementation would be as follows :

15:07 < kartikdutt18[m]> 1. If number of input features(input.n_rows) equal size parameter of the layer we can assume that it came from a linear layer and the current implementation would work.

15:08 < kartikdutt18[m]> 2. If they are equal, then we could perform mini batch norm (batch norm on each channel / feature map) for the input.

15:08 < kartikdutt18[m]> * 2. If they aren't equal, then we could perform mini batch norm (batch norm on each channel / feature map) for the input.

15:09 < KimSangYeon-DGU[> Is there any reference to implement that?

15:09 < kartikdutt18[m]> Sure. One second.

15:10 < sakshamb189[m]> I just have one doubt.

15:10 < kartikdutt18[m]> I think this might be [useful](https://gluon.mxnet.io/chapter04_convolutional-neural-networks/cnn-batch-norm-scratch.html).

15:11 < sakshamb189[m]> nvm..I think I might need to think it through for sometime

15:14 < KimSangYeon-DGU[> sakshamb189: Ahh, Ok, can you tell us the point you're thinking?

15:14 < sakshamb189[m]> yup I think it should work .. the approach looks fine

15:15 < sakshamb189[m]> This way we don't have to pass any additional image dimensions through the constructor

15:16 < kartikdutt18[m]> Right, we will be calculating mean for the a feature map, the width and height won't have any effect on the mean and variance.

15:18 < kartikdutt18[m]> I think it should be ready for review by tomorrow. The forward part is nearly complete and the backward and gradient won't change much.

15:19 < KimSangYeon-DGU[> And, when implementing that, one thing I want to say is that Armadillo uses column-major matrix and PyTorch uses row-major matrix.

15:22 < kartikdutt18[m]> Right.

15:25 < KimSangYeon-DGU[> Ok, is there anything you guys want to discuss?

15:33 < KimSangYeon-DGU[> Great. Looks good to me

15:33 < kartikdutt18[m]> Awesome. Once the batch norm layer is done we will be able to do that.

15:34 < sakshamb189[m]> sounds good

15:35 < KimSangYeon-DGU[> After loading the weights, will you save it to `.bin` file, right?

15:35 < kartikdutt18[m]> Yes.

15:35 < KimSangYeon-DGU[> Ok, sounds great

15:38 < KimSangYeon-DGU[> It seems our meeting is done :)

15:39 < sakshamb189[m]> okay then we'll meet next time. Have a nice weekend guys!

15:39 < KimSangYeon-DGU[> I think the project is going well

15:39 < sakshamb189[m]> I'll review the batchnorm PR tomorrow.

15:39 < KimSangYeon-DGU[> Thanks, guys

15:40 < KimSangYeon-DGU[> Yes, have a great weekend!

15:40 < kartikdutt18[m]> Great. Have a great week @KimSangYeon-DGU , @sakshamb189

15:41 < KimSangYeon-DGU[> But, we should be careful until succeeding to convert the weight and verify it works.

15:43 < kartikdutt18[m]> <KimSangYeon-DGU[ "But, we should be careful until "> Agreed. Logically, I think it should work. Provided the weights are same I can't think of a reason why it shouldn't.

15:43 < KimSangYeon-DGU[> Awesome!

15:44 < kartikdutt18[m]> Great.

15:47 < kartikdutt18[m]> Ahh, found this PyTorch to Keras weight converter [here](https://github.com/nerox8664/pytorch2keras). I think we can include this in PoC.

15:47 < kartikdutt18[m]> Also, I will update [this repo](https://github.com/kartikdutt18/mlpack-PyTorch-Weight-Translator) tomorrow with the python part of the code.

15:48 < KimSangYeon-DGU[> Ok :)

16:10 < shrit[m]> rcurtin: Can you give me your thoughts on this: https://github.com/mlpack/mlpack/pull/2459/files/9a113c3c34b0d304d90cd9a759d8927e234f8bff..6e9f5eed0cd89a4e30425b9148a9d961d2699c2f#diff-7263eb5a1abede39add281b539cfb0c2 ? thanks

16:55 < rcurtin> shrit[m]: sure, I saw this but my world is a bit on fire today (not literally!) so I haven't had a chance to review the CLI11 PR

16:55 < rcurtin> I think we need to avoid constexpr though because we can't count on a C++17 compiler, unfortunately

16:56 < rcurtin> are you sure constexpr is needed there?

16:56 < shrit[m]> Yes, I was not able to compile without it

16:56 < rcurtin> oh this is an MSVC error, I see from the StackOverflow post

16:56 < shrit[m]> I got very similar error to this on Stackoverflow post

16:56 < rcurtin> actually I think this would not be specific to MSVC

16:57 < rcurtin> yeah, so unfortunately I guess our only choice is to leave it in templated functions :(

16:57 < rcurtin> sorry that my suggestion was not correct! :(

16:58 < shrit[m]> You do not have to worry about, I thought you might have another idea since you are more expert than me in SFINAE

16:58 < shrit[m]> I will keep the both functions and test only for bool

16:59 < shrit[m]> The day we move to C++17 we will use these lambdas

17:00 < rcurtin> agreed!

17:00 < rcurtin> unfortunately legacy OS support probably means that day will be in... 2027 :-D

17:01 < shrit[m]> Hope this will be 2022

17:05 < rcurtin> hehe, Ubuntu LTS and RHEL are what holds us back :)

17:06 < rcurtin> although it looks like RHEL8 has gcc 8, which has basically everything we'd need (unlike RHEL7, which we should still actively support, which has the ancient gcc 4)

20:56 < shrit[m]> how long do you think it is still going to be used RHEL 7?

20:56 < shrit[m]> is it still maintained ?

20:56 < rcurtin> RHEL7 EOL is 2024 :-D it is in wide use in large companies, university systems, etc.

21:04 < shrit[m]> I never used RHEL, I am not sure if it is well adapted in France, people here use mostly debian,

21:05 < shrit[m]> I have seen only one university use Fedora, this is the closest thing to RHEL

21:05 < rcurtin> that must be nice, personally I prefer debian :)

21:12 < shrit[m]> it is true debian is good, I used to use debian testing, but I had to resolve a lot of dependecy issue.

21:13 < shrit[m]> Then I have switched to Arch, and since then I found the most suitable distro for me

21:24 < rcurtin> :)