rcurtin_irc changed the topic of #mlpack to: mlpack: a scalable machine learning library (https://www.mlpack.org/) -- channel logs: https://libera.irclog.whitequark.org/mlpack -- NOTE: messages sent here might not be seen by bridged users on matrix, gitter, or slack
kristjansson has quit [Ping timeout: 240 seconds]
kristjansson has joined #mlpack
Guest11 has joined #mlpack
Guest11 has quit [Quit: Client closed]
<AnwaarKhalid[m]> <zoq[m]> "Hello everyone, I think I went..." <- rcurtin: could you also try to look at the doc please?
<rcurtin[m]> yeah, I am traveling. I have a few hours now while I wait for my inefficient friends to wake up 😄 I'm working through some emails now... I will try to get to yours before I leave, but don't wait for my input if I can't get to it in time
<shrit[m]> rcurtin[m]: untill now !! 😄
<GopiMTatiraju[m]> rcurtin: zoq I know I am late but I am working on Bandicoot proposal, but since seeing the last mail on the mailing list, I just wanted to know that are you guys interested in mentoring the project? I want to work on it, and last with parser also we were trying to bring those design changes that we need for bandicoot. Also I was planning to apply for SDG so maybe if we can get started here we will be on a good road...
<GopiMTatiraju[m]> I am targeting that we should be atleast able to run the simple MNIST example on GPU by the end of the project
<GopiMTatiraju[m]> And benchmarking ofcourse
<zoq[m]> GopiMTatiraju[m]: GSoC proposal you mean?
<GopiMTatiraju[m]> Yuppp
<GopiMTatiraju[m]> Long project
<GopiMTatiraju[m]> Fits the size I guess
<zoq[m]> I'm still 100% interested, and I think the scope is fine as well, as long as we focus on a specific part of the codebase e.g. ann.
<GopiMTatiraju[m]> Yupp, ANN is a good starting point.
<GopiMTatiraju[m]> I was thinking maybe these can be the base points... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/e0dc7e9c7dfa08119e7e1fc40f78c518f25f485b)
<GopiMTatiraju[m]> Also, should we target a single backend or multiple backends?
<zoq[m]> > <@gopitatiraju:matrix.org> I was thinking maybe these can be the base points... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/41d5505cdd90f17c41f75e364a7679e35c0e524e)
<zoq[m]> zoq[m]: We might have to be careful with the benchmark part, depending on the focus of the project, we don't optimize the kernels?
<GopiMTatiraju[m]> Yup, I might make a PR soon for changes in the parser, we will need that one as well...
<zoq[m]> GopiMTatiraju[m]: Might be reasonable, I mainly focused on CUDA.
<GopiMTatiraju[m]> Yupp, I also want to work on CUDA mainly...
<GopiMTatiraju[m]> Do we need any modifications from ensmallen's side?
<zoq[m]> GopiMTatiraju[m]: not anymore
<GopiMTatiraju[m]> Oh thats great there, I will start of by making a list of class which we need to work on
<GopiMTatiraju[m]> <zoq[m]> "We might have to be careful with..." <- Considering the timeline I think we can come-up with good kernels.
<GopiMTatiraju[m]> I remember you were saying that first thing is to come up with a simple kernel and then start working on it's optimisation using tools like NVIDIA profiler
<GopiMTatiraju[m]> I am also reading the Learn CUDA by examples book which you suggested.
<zoq[m]> > <@gopitatiraju:matrix.org> Considering the timeline I think we can come-up with good kernels.
<zoq[m]> > I remember you were saying that first thing is to come up with a simple kernel and then start working on it's optimisation using tools like NVIDIA profiler
<zoq[m]> Depending on the kernel, you can do some clever things to get some huge optimizations.
<GopiMTatiraju[m]> Ohh, sounds fun, do you have any examples of like something that I can read up maybe?
<GopiMTatiraju[m]> coot::Mat is nearly done, right?
<zoq[m]> GopiMTatiraju[m]: For the `accu` and `dot` rcurtin followed https://developer.download.nvidia.com/assets/cuda/files/reduction.pdf which is a great resource; it covers different approaches.
<zoq[m]> GopiMTatiraju[m]: In the sense of base operations like `%=` and `+=` yes.
<GopiMTatiraju[m]> Okay, I will read it.
<GopiMTatiraju[m]> We need to revamp the whole `FFN` class right?
<GopiMTatiraju[m]> Atleast for the starters
<GopiMTatiraju[m]> Also I can see that there are some functions which are not templatized, what do we do with those?
<GopiMTatiraju[m]> s/templatized/templetized/
<zoq[m]> > <@gopitatiraju:matrix.org> Okay, I will read it.
<zoq[m]> > We need to revamp the whole `FFN` class right?
<zoq[m]> The FFN class should be fine, the main focus is on the layer implementation.
<zoq[m]> > Atleast for the starters
<zoq[m]> is one example that needs to be refactored
<zoq[m]> it uses the correct element type, but `arma::Cube<..>` internally.
<zoq[m]> is another example, for one the `transform` function is not implemented in bandicoot, and supporting anonymous lambda functions is not really something we can add.
<GopiMTatiraju[m]> There will be places where we need to make more than just `arma -> coot` changes....
<GopiMTatiraju[m]> I was thinking these layers should be most basic ones... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/e393ded65b33e6831aa6a44e9543f25cd5512ad1)
<zoq[m]> Yeah, might be a good idea to look into the examples folder to figure out what layers are used for existing models. `BatchNorm` should be important as well.
<GopiMTatiraju[m]> Yupp, looking there only...
<GopiMTatiraju[m]> I took these layers from Simple MNIST [example](https://github.com/mlpack/examples/blob/master/mnist_simple/mnist_simple.cpp)
<GopiMTatiraju[m]> I will add BatchNorm to the list as well, need to see what all arma functions these layers are using now...
<zoq[m]> Looking into https://github.com/mlpack/models would be useful as well.
<GopiMTatiraju[m]> Yupp, having a model trained on GPU makes more sense...