05:19 < JaraxussTong> apir8181 : sorry for the late response. I use quassel irc client, it is a distributed irc software.I set the core of quassel on my server in sanfrancisco purchased in digital ocean.

13:28 < apir8181> Hi, is armadillo has some function like matlab bsxfun?

13:30 < apir8181> it just has operation each_col or each_row..

13:53 < zoq> apir8181: I think imbue and transfrom is close to the bsxfun function.

13:55 < zoq> apir8181:: Btw. Did you get the message from JaraxussTon?

13:56 < apir8181> oh.. I just want to have broadcast add a vector to a matrix.

13:57 < apir8181> Not yet. I have search through the channel logs. It is I miss something here?

14:00 < apir8181> What about the operation arma::join_rows, is it costly (due to copy of the original matrices)?

14:01 < apir8181> I just want to add intercept (bias) term to a data matrix.

14:02 < apir8181> I'm trying to fixed softmax regression bug (issue 332).

14:12 < zoq> "JaraxussTon> | apir8181 : sorry for the late response. I use quassel irc client, it is a distributed irc software.I set the core of quassel on my server in sanfrancisco purchased in digital ocean."

14:17 < apir8181> aha, thanks :)

14:17 < zoq> Ideally join_rows uses the mem_ptr from the original matrix.

14:17 < apir8181> So, there is no copy cost in this operation?

14:22 < zoq> apir8181: Maybe we need to check the source code or maybe naywhayare knows the answer :)

14:24 < naywhayare> apir8181: join_rows is generally expensive

14:25 < naywhayare> it does incur a copy of the entire matrix

14:25 < naywhayare> I just saw your update of ticket #322

14:25 < naywhayare> the missing intercept is indeed the problem, and at one point I had a patch that fixed it, but I'm not sure where it is now

14:25 < naywhayare> let me see if I can dig it out. it needed to be reworked a little bit, if I remember right...

14:29 < naywhayare> https://gist.github.com/rcurtin/1cf62c411e0215f1d4b4

14:29 < naywhayare> those are the changes I made, but I don't think it's done quite yet

14:30 < naywhayare> maybe you can reuse the changes there? if not, no big deal :)

14:30 < naywhayare> either way, that patch probably needs some additional documentation about the intercept being a part of the parameters matrix

14:32 < apir8181> OK, I will try.

14:35 < apir8181> I could not fully understand where you add the intercept term.

14:36 < apir8181> why not change this line [ hypothesis = arma::exp(parameters * data); ] ?

14:37 < apir8181> and this line [ parameters.randn(numClasses, inputSize); ] to [parameters.randn(numClasses, inputSize + 1);] ?

14:42 < apir8181> In my implementation, I change the parameters size and compute the gradient of parameter.col(0) specially.

14:50 < naywhayare> apir8181: sorry, I stepped out

14:50 < naywhayare> I ran some tests... yeah, my patch doesn't actually fix the tests (as they are written now)

14:50 < naywhayare> so maybe it is not useful...

14:51 < apir8181> so what is this patch for?

14:52 < naywhayare> I can't remember, it was so long ago :(

14:52 < naywhayare> the patch is just the modified files I had in a working directory

14:52 < naywhayare> I remember working with Siddharth to isolate the issue, and I remember modifying the code to fix it, and thinking "it's not quite done yet..."

14:53 < naywhayare> so maybe I didn't actually manage to fix it and the patch is useless...

14:56 < apir8181> aha. BTW, it seems that many machine learning project are rejected in GSOC this year.

14:57 < apir8181> for what I know, shogun, scikit-learn, mlpack, xapian

14:58 < naywhayare> did scikit-learn apply?

14:58 < naywhayare> I was talking to the shogun guys a few days ago about it

14:58 < naywhayare> I met them at the summit last year... cool guys

14:58 < naywhayare> I think maybe one reason is that only 137 organizations were accepted this year, as opposed to 190 last year, and something like 80 organizations were new this year

14:59 < naywhayare> so maybe they just wanted to try something new? I don't know

16:51 < apir8181> With the same regularization value, it seems that softmax regression regularization term is more constrained than logistic regression.

16:51 < naywhayare> apir8181: are you sure? in the two-class case, the derivation of softmax regression reduces exactly to logistic regression

16:53 < apir8181> Let me check again.

16:55 < naywhayare> I have to go get lunch, but I'll be back in about an hour

16:55 < naywhayare> http://ufldl.stanford.edu/wiki/index.php/Softmax_Regression

16:55 < apir8181> softmax: norm(\theta_0, 2) + norm(\theta_1, 2); logistic: norm(\theta, 2) = norm(\theta_0 - \theta_1, 2)

16:55 < naywhayare> those are the notes I used when I did my derivation to check that softmax is the same as logistic when the number of classes is 2

16:56 < naywhayare> hm, I don't think I thought about the regularization term

16:56 < apir8181> sorry, I forget the ^2 term.

16:56 < naywhayare> I'll take a look at this when I get back

16:57 < apir8181> I have to go to sleep. Have a good lunch.

18:50 < stephentu> naywhayare: ive got my irc setup rolling now

19:22 < naywhayare> :)