07:24 < davida> zoq: As a way to take advantage of the speed of the cube multiplication, yet still be flexible enough to accept arbitrary sequences, rather than modifying Rho with each step, would it make sense to take in an extra masking cube. Alternatively, feeding in a rowvec of Rho values might be useful. Would need to take care of the "Shuffle" in the optimizer though.

12:09 < zoq> davida: I think either we provide another vector with the length or we use arma::field.

13:33 < Hemal> Hello, I am trying to understand the MLpack codebase, so that I will be able to contribute, Currently I am looking at https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/neighbor_search/knn_main.cpp

13:33 < Hemal> I don't understand the terms 'PROGRAM_INFO' 'PARAM_MATRIX_IN', 'PARAM_MATRIX_OUT'

13:33 < Hemal> What are they? They don't seem to be macros, since macros use #define directive?

13:38 < rcurtin> Hemal: they are macros to describe the parameters that the mlpack_knn program will use

13:39 < rcurtin> they are defined in src/mlpack/core/util/param.hpp

13:45 < Hemal> ok, i will check it out.

13:47 < Hemal> Any tips on getting me started, I've set up the dev environment, read the style guidelines ,run knn and now trying to understand the code base. After understanding the codebase, i would move to PRs for contribution.

14:05 < rcurtin> Hemal: sounds good, it looks like you have done most of what I would recommend

14:05 < davida> I saw earlier in the year there was a question about L2 Regularisation which was not yet implemented in MLPACK. Is there a plan to add this?

14:08 < rcurtin> davida: L2 regularization for neural networks? or something else?

14:09 < davida> Yes, for NN.

14:09 < rcurtin> ok, I see. I don't know of anyone actively implementing it, but it shouldn't be too hard to do

14:10 < Hemal> rcurtin that sounds great :)

14:40 < davida> rcurtin: I believe L1 and L2 regularization are both available in keras.

14:40 < davida> from keras import regularizers

14:40 < davida> model.add(Dense(64, input_dim=64, kernel_regularizer=regularizers.l2(0.01)

14:45 < rcurtin> yep, they are; it's a little unclear to me what the best way is to add it to mlpack's framework. Maybe modifying the FFN and RNN classes directly is an option, but it might also be possible to do it through the proposed 'FunctionWrapper' interface

14:46 < rcurtin> where basically you might do, e.g., 'model.Train(data, responses, std::make_tuple(PrintLoss<>(), L2Regularization<>(0.01), ModelCheckpoint<>()))'

14:46 < rcurtin> I'm not sure if I have the syntax there right, but basically this would be somewhat similar to the idea of Keras callbacks, except it will all be worked out at compile time so there should not be any overhead

14:49 < davida> sounds good.

14:56 < rcurtin> can't make any promises on how long it will take until that idea sees the light of day; I'm currently pretty underwater with the process of moving mlpack's optimization framework into its own library

14:56 < rcurtin> we got most of that done a couple weeks ago and the framework (ensmallen) is now available on its own, but the documentation needs some improvement so this is how I am spending my evenings this week...

15:56 < davida> No problem. Can use Dropout in the meantime.

15:57 < davida> Why did you want to move the optimizer framework out of MLPACK? Are ppl asking to have access to the optimizers and not the machine learning framewrok?

16:05 < rcurtin> davida: basically, yeah---the optimizer framework is useful on its own for lots of problems

16:59 < davida> quit