naywhayare changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
< jenkins-mlpack>
* saxena.udit: BiBTex added to adaboost_impl.hpp
< jenkins-mlpack>
* Ryan Curtin: Refactor to be stricter with types (arma::Col<size_t> instead of arma::vec).
< jenkins-mlpack>
* andrewmw94: R* tree split. Default to using the R* tree in allknn
< jenkins-mlpack>
* Ryan Curtin: Comment normalization.
Anand has quit [Ping timeout: 246 seconds]
oldbeardo has joined #mlpack
< oldbeardo>
naywhayare: hey, I read your comment
< oldbeardo>
I agree with the derivation, theoretically they are the same, but the learning skews the Softmaz Regression module
< oldbeardo>
*Softmax
< oldbeardo>
I believe this happens since you are learning two different parameters, and not a single parameter which will even things out
< naywhayare>
it's easy to re-adapt the derivation to show that the function being minimized has the same local minima for the two-class softmax regression case and for logistic regression
andrewmw94 has joined #mlpack
< naywhayare>
you can try constructing a LogisticRegression object with weights equal to theta_1 - theta_0 (where theta_1 and theta_0 come from the SoftmaxRegression object you trained on two classes)
< naywhayare>
and it should give the exact same classifications as the SoftmaxRegression object
< oldbeardo>
okay, could you try that for me? I think I have written most of the other tests, I'm trying out the test for the template specialization
< naywhayare>
I don't have time to try that right now
< naywhayare>
there is no huge hurry for softmax regression, so we can come back to that later
< oldbeardo>
okay, I'll try to find some time for it
< naywhayare>
like I said, no hurry. if it doesn't make it into the release, it's not a big deal; we can have another release once it's ready
< naywhayare>
I think releases should be far more often than every six months, so I'd like to try to start doing that after this release
< naywhayare>
hmm... maybe not "far more often", but probably every two to four months would be better
< oldbeardo>
right, by the way, how did your paper submissions go?
< naywhayare>
that was almost a month and a half ago now, I guess :)
< naywhayare>
they went reasonably well, I guess. I haven't heard back with any reviews yet
< oldbeardo>
sorry, I meant did any of those get published?
< naywhayare>
not yet; I can email you what I submitted, though
< naywhayare>
if you are interested
< oldbeardo>
sure, I would like to read it
< naywhayare>
it's a double-blind review, though, so it's somewhat important that I don't publish the PDFs somewhere online; then, a reviewer could find them and figure out who I was; then the review would be biased, etc., etc.
< oldbeardo>
yeah, I'm aware of the procedure, I will be using it only for my own viewing, and I'm certainly not a reviewer :)
< naywhayare>
:)
< naywhayare>
sent
< oldbeardo>
thanks
udit_s has joined #mlpack
govg has joined #mlpack
govg has quit [Changing host]
govg has joined #mlpack
govg has quit [Ping timeout: 245 seconds]
oldbeardo has quit [Ping timeout: 246 seconds]
oldbeardo has joined #mlpack
< oldbeardo>
naywhayare: I added tests for Reg SVD, please have a look
< jenkins-mlpack>
Starting build #2037 for job mlpack - svn checkin test (previous build: SUCCESS)
oldbeardo has quit [Quit: Page closed]
andrewmw94 has quit [Ping timeout: 264 seconds]
andrewmw94 has joined #mlpack
< marcus_zoq>
naywhayare: It is neccessary to include the kernel header files into the fastmks_main.cpp file?
< naywhayare>
marcus_zoq: hm, I don't think it is anymore; all the kernels appear to be included in core.hpp
< naywhayare>
marcus_zoq: do you think we should use 'ö' in the documentation for kernel_pca? we could do that since it's just documentation, if you wanted :)
< naywhayare>
you would be the expert here, not me :)
udit_s has quit [Read error: Connection reset by peer]
< jenkins-mlpack>
siddharth.950: Adding tests for Reg SVD.
< jenkins-mlpack>
Starting build #2038 for job mlpack - svn checkin test (previous build: SUCCESS)
< marcus_zoq>
naywhayare: I'm not sure what you mean ... Did I write 'ö' -> 'oe' (encoding...) somewhere?
imi has joined #mlpack
< naywhayare>
marcus_zoq: oh, sorry. you'd written 'oe' in the documentation, but I figured maybe it might be more correct to write the actual ASCII 'ö' in that case
< naywhayare>
I suppose you could use ö in the actual class name too (I think C++ would allow that) but then everyone without an ö on their keyboard (including me) would have a terrible time working with that code :)
< naywhayare>
anyway, it would not be wrong to say I have no idea what I am talking about here :)
udit_s has quit [Quit: Leaving]
< imi>
hello. let's suppose I have a binary classification problem where the optimal solution (for me) would be not the maximal probability (ratio among trainingset examples) of good answer, but I'd like to weight my trainingset examples, so a single mistake for one of the examples would be not as bad as a single mistake for an other? can it be done using machine learning? is there any better solution for a custom "fitness function" than
< imi>
genetic programming?
< naywhayare>
imi: often you can adapt the learning algorithms to weight particular examples in a certain way
< naywhayare>
this is what AdaBoost does
< naywhayare>
and fortunately, we have a guy implementing AdaBoost...
< naywhayare>
AdaBoost is an ensemble method, so it depends on a handful of weak learners and it requires that each of those be able to learn examples with weights
< naywhayare>
right now the perceptron is all that is implemented in this way, though, so that is probably the only thing that will work out-of-the-box
< naywhayare>
another possibility is to weight your "important" examples by duplicating them in the dataset the correct number of times to get the right distribution
< naywhayare>
but then you can end up with rather large datasets...
< naywhayare>
it isn't too hard to extend other gradient-descent-type methods to support individual example weights
< naywhayare>
for instance, if you look at logistic_regression_function.cpp, you could modify lines 72-75 and lines 101-105 to multiply against a weight for the particular point (whose index is i)
< naywhayare>
you'd have to do a bit of refactoring at higher levels (i.e. the LogisticRegressionFunction constructor and the LogisticRegression constructor), but I think that idea could work for logistic regression