06:06 < hodor12345678> Hello Everyone,I am new to OSS and want to contribute to get proposal in GSoC.Anyone please guide me.

12:34 < alexscc> one thing I’ve noticed is that running tests on 8 core, there are warnings about timers already being started

12:34 < alexscc> is the framework fully reentrant, besides logging?

14:10 < hodor12345678> Hello everyone,I want to contribute to mlpack for GSoC 2018.Where can I start?Somebody please help me.

14:17 < zoq> hodor123456: Hello there, www.mlpack.org/involved.html might be helpful.

14:18 < zoq> and since you mentioned GSoC http://www.mlpack.org/gsoc.html might be interesting too.

14:19 < hodor12345678> Thank you,Can I read history chat of IRC channels?

14:19 < zoq> btw. great name, reminds me off GoT

14:19 < zoq> yes: http://www.mlpack.org/irc/

14:20 < zoq> make sure to checkout the mailinglist as well: http://knife.lugatgt.org/pipermail/mlpack/

14:22 < zoq> alexscc: Did you run the test against the master branch?

14:22 < alexscc> zoq: brew installed

14:23 < zoq> alexscc: I think it should be solved with: https://github.com/mlpack/mlpack/pull/1135

14:24 < alexscc> ok it’s fixed then, nice

14:25 < alexscc> btw, who implemented NCA? I have some questions about usage of the parameters, I am going a bit random atm here… if I set number of iterations = 0, what should be the tolerance in order to see it converging within, say, minutes?

14:29 < alexscc> my data is 71-dimensional, 15 classes more or less, 100 samples per class

14:30 < alexscc> I’d basically like to know how to decide the tolerance parameter: for example, does it depend on wether one starts the gradient descent with a normalization matrix?

14:34 < hodor12345678> alexscc thanks,btw GOT is love.

14:35 < hodor12345678> sorry,I meant zoq

14:35 < zoq> alexscc: Ryan implemented NCA, so he could probably provide some insights on this, probably not today since it's thanksgiving.

14:36 < zoq> If you use standard SGD as optimizer you could test different step sizes; and not sure setting iterations = 0 is necessary, since it's not granted to converge in the given time.

14:36 < alexscc> just launched a 500000 iteration

14:37 < alexscc> let’s see if it converges within some time

14:37 < zoq> hodor123456: Yeah awesome books and great show.

14:38 < zoq> alexscc: We recently added batch support, which could accelerate the training process.

14:38 < alexscc> cool is there any place where I can read about it?

14:38 < zoq> alexscc: It should at least converge after the given number of iterations.

14:39 < alexscc> yup

14:39 < zoq> alexscc: https://github.com/mlpack/mlpack/pull/1137

14:40 < zoq> you have to build against the master branch

14:40 < zoq> Ryan did run some tests using a simple neural network: http://www.ratml.org/misc_img/batch_size_sweep.png

14:41 < alexscc> super

14:42 < alexscc> I’ll use the master branch from now on

14:43 < zoq> batch support is going to be part of mlpack 3.0

14:44 < zoq> but until then, the master branch is probably the best option

22:00 < alesc> I guess I’ll plot the difference between objectives to understand how training is going. on Monday :)

22:06 < alsc> please let me know if you have suggestions about how to treat NCA.. nca_iterations: 30000 nca_stepSize: 0.0001 nca_tolerance: 0.0001 took quite long and went to 91% from 89% accurcay over basic KNN/. I am sure I can get it better

22:14 < rcurtin> alsc: your nick keeps getting shorter :)

22:14 < rcurtin> sorry for the lack of responses, I am on vacation right now

22:14 < rcurtin> are you using the l-bfgs or SGD optimizer? if I am remembering correctly, the L-BFGS optimizer is able to take advantage of some significant optimizations that can make it orders of magnitude faster

22:15 < rcurtin> zoq is right that now using larger batch sizes will help accelerate it, but the problem is that optimizing the NCA objective requires full calculation of the denominator,

22:15 < rcurtin> even if you are just calculating the objective (or gradient) for a single point or a few points

22:16 < rcurtin> since the parameters change each iteration, that denominator has to be fully recalculated (which is O(N) time) so taking a small step with SGD takes the same amount of time as L-BFGS

22:16 < rcurtin> and since L-BFGS typically needs to take far fewer steps, it ends up being much faster

22:17 < rcurtin> I don't have any input for how to set the tolerance parameter really

22:17 < rcurtin> I believe that as the objective decreases, KNN performance *should* (but is not guaranteed to) improve

22:18 < rcurtin> to be perfectly honest, I think NCA is a nice metric learning technique, but I think it can be significantly outperformed by simpler techniques

22:18 < rcurtin> a long time ago I wrote an algorithm that was a metric learning technique that might be more applicable, found in this (not very good) paper: http://ratml.org/pub/pdf/2011mlsp.pdf

22:19 < rcurtin> like I said I don't think the algorithm there is particularly good, but it can be quicker than NCA and produce better results

22:19 < rcurtin> I never applied it to other tasks and never polished it for an mlpack implementation (whatever code I have now surely wouldn't compile)

22:20 < rcurtin> but all this rambling is to say that I think the NCA technique of "make a convex objective function so we can use well-known optimizers" may not be as good a technique as something more straightforward, like

22:21 < rcurtin> "let's use a simulated annealing-like algorithm to randomly guess a better metric for KNN"

22:23 < rcurtin> I think there is still a lot of interesting work to be done in the field of metric learning, but I think people are less interested in it and more interested in deep learning these days :)

22:23 < rcurtin> ok, that's enough rambling for now :)

22:23 < rcurtin> I make no guarantees that my opinions or postulations are correct :)