07:37 < nilay> zoq: Hi, now I can complete training for the test net with one sample which uses inception layer. Like you said we use a small test with small input to check if forward output "looks correct". But how do we ensure the output looks correct?

10:34 < zoq> nilay: Hello, you can set the weights to some static value and compare it with a reference model or your own calculation.

10:50 < nilay> ok

11:06 < nilay> zoq: does the Gradient method of inception layer look correct?

11:43 < zoq> nilay: at first glance, it looks good

12:12 < nilay> but then isn't this a problem for concat_layer, because we are only taking the base layers to concatenate, but we need the delta we have in the concat_layer for the gradient function of the bias_layer (which is below the base_layer)

14:38 < zoq> nilay: I'm not sure I get the problem, if the network looks like {bias, concat(A, B)}, the concat layer passes the delta to A and B.

14:42 < nilay> zoq: are you saying i can pass the delta to the base layer and it'll all get handled

14:43 < zoq> If I didn't missed anything, yes

14:46 < nilay> zoq: so i have 4 base layers that i am passing to the concat_layer to concatenate them. Now when i am updating the gradient, i have the delta in the concat layer, but i want to be able to pass the delta to the bias layers which are under these base layers (only base_layers are only provided to us)

14:46 < nilay> do you see the problem?

14:50 < zoq> I'm not sure, so the concat layer is basically a BaseLayer but it take a bunch of layers as input right?

14:54 < nilay> it does not do any operation, it takes a bunch of layers as input, aggregates there output (forward pass), distribute errors (backward pass) and delta (gradient).

14:56 < zoq> So it returns the delta which can be used for the backward pass for the before the concat layer, I'm sorry I don't see a problem here, maybe you can write some proof of concept?

14:57 < zoq> I would recommend to finish the inception layer first, before moving on with the concat layer.

14:57 < nilay> inception layer is finished we just need to the small test you talked about

14:59 < zoq> in this case, I think we should finish that test first, and open a PR to get the code merged in.

14:59 < zoq> writing some good tests is always tedious work :(

15:02 < nilay> ok then, i'll write the test, one question, do we want to merge this code only or write a subnetwork_layer which can be duplicated better?

15:04 < zoq> I think we should first merge the inception layer, if it turns out a subnetwork would make things easier, we can open another PR.

15:04 < nilay> for the proof of concept about the previous discussion, inception layer does just that, we have a delta available to us in the Gradient function, now we use that delta not in the base layer but the bias layers which are under that base layer

15:06 < nilay> do people use subnetwork (layers like inception layer which are duplicated many times) in cnns today?

15:07 < zoq> But in the inception layer we do it manually, and if the the bias layer is not in the layer the surrounding cnn takes care of it.

15:07 < zoq> You mean inception layer A -> inception layer B -> ... -> inception layer N?

15:09 < nilay> yeah subnetwork repeated many times in net, but may also have some other layers in between.

15:09 < zoq> yes, that's pretty common

15:10 < nilay> so such a layer would be good

15:11 < zoq> Yes, I think that would be a great addition.

15:11 < zoq> But it should work with every layer not just the inception layer.

15:11 < nilay> yes thats what i meant

15:14 < nilay> so i guess i will write the test first, and we'll discuss concat_layer later as i am not sure i got what you mean to say.

15:14 < zoq> sounds good

16:29 < Aterons> Could anyone here spend a few minutes answering a complete newbi question about the Cpp API ?

16:40 < zoq> Aterons: Hello, we are here to help, don't hesitate to ask questions :)

16:59 < Aterons> Hmh, I had thought I figured it out, but apparently not. I guess what I should ask really is: Should I be able to construct MLpack obejcts using mat/vectors declared "inside" the program or do they need to be loaded from CSV files ?

16:59 < Aterons> eg if I have "LinearRegression linR(d, r);"

17:00 < Aterons> and I declare the response vector as vec responses = { 1, 3, 5, 10, 17 }; ?

17:00 < Aterons> is that correct ? Or should I use some loadVecotr function to load those values from a file ?

17:06 < zoq> Aterons: You can either load your parameter e.g. CSV or declare the parameter: Take a look at line 72 in https://github.com/mlpack/mlpack/blob/master/src/mlpack/tests/linear_regression_test.cpp