ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
< rcurtin>
it includes me (and others) warming up karts, then qualifying, then finally starting the race (I am the second-from-last qualifier, then I ran a lap that put me in 3rd place for the start)
< rcurtin>
the race actually starts at 8:15 or so, the rest might be pretty boring :)
xiaohong has joined #mlpack
< rcurtin>
robertohueso: I read through the paper, and it seems to me like this is most easily thought of from the bottom up instead of the top down
< rcurtin>
so, e.g., a KD-tree gets built; then, we look at the leaf nodes and run PCA to recover d eigenvectors, and those constitute our basis for that leaf node
< rcurtin>
in the figure, you're right, d = 1, but I think it would be possible to choose a greater d
< rcurtin>
then, we can look at the parent nodes of those leaf nodes and use the algorithm given in [8] to merge the two sets of eigenvectors
< rcurtin>
and do this the rest of the way up the tree
< rcurtin>
it seems to me like you would also need to hold the projections of the points (not the descendants, just the points I think) in each node's statistic
< robertohueso>
That makes sense to me :) I was kinda confused by that, so yeah a node statistic should be enough. And yeah, I also think we need to hold the projections
< robertohueso>
Thanks for the clarification!
< rcurtin>
of course :)
< rcurtin>
robertohueso: ran out of time tonight for the RectangleTree copy constructor... I can do it in the next days, so you can focus on the subspace tree :)
k3nz0_ has quit [Ping timeout: 245 seconds]
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Read error: Connection timed out]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
sreenik[m] has quit [Ping timeout: 250 seconds]
Sergobot has quit [Ping timeout: 265 seconds]
chandramouli_r has quit [Ping timeout: 265 seconds]
aleixrocks[m] has quit [Ping timeout: 265 seconds]
xiaohong has joined #mlpack
KimSangYeon-DGU has joined #mlpack
chandramouli_r has joined #mlpack
Sergobot has joined #mlpack
aleixrocks[m] has joined #mlpack
sreenik[m] has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
k3nz0_ has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
sumedhghaisas has joined #mlpack
< KimSangYeon-DGU>
sumedhghaisas: Hi Sumedh, I'm ready and I'm sorry to hear that you are sick :(
< sumedhghaisas>
KimSangYeon-DGU: Hey Kim.
< sumedhghaisas>
feeling better already
< sumedhghaisas>
how have you been?
< KimSangYeon-DGU>
I set theta as a trainable variable
< KimSangYeon-DGU>
tested it so far
< sumedhghaisas>
ahh yes had couple of question about that as well.
< sumedhghaisas>
hows it looking?
< KimSangYeon-DGU>
Umm, I think it's sensitive to lambda
< KimSangYeon-DGU>
I'll write a document for that
< KimSangYeon-DGU>
I just set theta as a trainable scalar variable.
< sumedhghaisas>
yes I was just going to say that
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
cause the equation restricting that depends on it
< sumedhghaisas>
so just to clarify
< KimSangYeon-DGU>
yeah
< sumedhghaisas>
before you were using equation under Equation 13 to compute cosine correct?
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
cool. And you removed it and used theta as a traineable parameter.
< sumedhghaisas>
But you are still using the 2 cluster case right?
< KimSangYeon-DGU>
Yeah, I'm using 2 clusters
< KimSangYeon-DGU>
Sumedh, I have a question
< sumedhghaisas>
Sure go ahead
< KimSangYeon-DGU>
I'm not sure I understand that "before you were using equation under Equation 13 to compute cosine correct?"
< sumedhghaisas>
ahh I meant how were you computing cosine before?
robertoh1eso has joined #mlpack
< KimSangYeon-DGU>
Ahh, right I used the equation in the constraint
< KimSangYeon-DGU>
under (13)
< KimSangYeon-DGU>
I understand :)
< sumedhghaisas>
yeah thats what I meant no worries :)
< KimSangYeon-DGU>
As you say, I removed it and change it to trainable variable
< sumedhghaisas>
okay so lets state the status so we can get some idea how to move on. Stop me if you think I am saying anything wrong
< sumedhghaisas>
We are currently doing experiments we 2 cluster case
< sumedhghaisas>
pertaining to Equation 13 in the paper
< sumedhghaisas>
we optimize NLL + lambda * constraint
< sumedhghaisas>
we observed that constraint optimization is not amazing good so we need to tinker with lambda or shift to another optimization al together
< sumedhghaisas>
tinkering lambda seems to work better
robertohueso has quit [Ping timeout: 246 seconds]
< sumedhghaisas>
and now we changed theta as traineable parameter and you are going to document the results of that in a new document
< sumedhghaisas>
so far so good?
< KimSangYeon-DGU>
Exactly :)
< sumedhghaisas>
great. Okay we also observed that alpha goes to zero for some cases but it seems to me from your lambda change that problem has been solved?
< KimSangYeon-DGU>
Right
< sumedhghaisas>
awesome
< sumedhghaisas>
lets also put that in the document :)
< KimSangYeon-DGU>
Got it
< sumedhghaisas>
very important observation indeed
< KimSangYeon-DGU>
I agree
< KimSangYeon-DGU>
I'll write in details
< sumedhghaisas>
so the root of the problem is unconstrained optimization it seems
< KimSangYeon-DGU>
Yes
< sumedhghaisas>
so that is the conclusion of the 2 directions we followed
< sumedhghaisas>
okay last thing
< sumedhghaisas>
so for some cases lambda = 1 actually does the correct thing right?
< KimSangYeon-DGU>
Yeah,
< sumedhghaisas>
Could you try those with lambda 53 and lambda 153 or something
< KimSangYeon-DGU>
I just tested it using lambda = 1
< sumedhghaisas>
I just want to make sure high value of lambda doesn't break the case
< KimSangYeon-DGU>
Ahh, I'll test it
< sumedhghaisas>
basically we tried the failed cases with lower and higher
< sumedhghaisas>
we should also check if higher value of lambda doesn't break normal cases
< KimSangYeon-DGU>
Yeah I tried but the result is not good than 53
< sumedhghaisas>
sorry didn't get that. you mean lambda 53 breaks the normal case?
< KimSangYeon-DGU>
I used lambda=53 in case lambda = 1 has bad result.
< KimSangYeon-DGU>
Lambda = 53 has good result than lambda = 1 in some cases
< KimSangYeon-DGU>
In research of 'Validity of the objective function', I found some cases result in bad results, so I tried to train by changing lambda. and it is the document 'Lambda impact'
< sumedhghaisas>
correct
< sumedhghaisas>
I was referring to the cases that lambda = 1 has good results
< KimSangYeon-DGU>
By the time, I wrote the two documents with theta that paper presents, not trainable variable. Thus, I think the result would be different if I test it with trainable theta
< KimSangYeon-DGU>
Yeah :)
< KimSangYeon-DGU>
Right
< sumedhghaisas>
Surely. We should create new documents for the traineable parameter changes. Take your time with the results. :)
< KimSangYeon-DGU>
*By the way
< KimSangYeon-DGU>
Yeah :)
< KimSangYeon-DGU>
Hmm, about the lambda, how can I find the best value of it?
< sumedhghaisas>
hmm yeah. always a problem with hyperparameters
< KimSangYeon-DGU>
Right
< sumedhghaisas>
there is no ideal way for it
< sumedhghaisas>
we can argue for now that there exists a good lambda such that it can be achieved
< sumedhghaisas>
we can impove it later
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
thats why I asked you to test the good cases with high lambda
< sumedhghaisas>
tht way we can say that for now high lambda works the best
< KimSangYeon-DGU>
That makes sense. I'll write documents about that.
< sumedhghaisas>
nice. okay another thing. I think we are at the point that we can also look at multiple cluster case
< sumedhghaisas>
I think we now have enough experience for it
< KimSangYeon-DGU>
So far, I observed too high lambda interferes with the training process
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
I'll write it with the exact graph and values
< KimSangYeon-DGU>
for clarification
< KimSangYeon-DGU>
I agree
< sumedhghaisas>
Great. Looking forward for the results. :)
< KimSangYeon-DGU>
Will test the multiple clusters as well
< sumedhghaisas>
I am still looking into how can I make the constrained optimization better I have some ideas
< sumedhghaisas>
but they might need some work still
< sumedhghaisas>
we have lot of options
< sumedhghaisas>
:)
< KimSangYeon-DGU>
Wow :)
< KimSangYeon-DGU>
I want to make more progress, so feel free to ask them :)
< sumedhghaisas>
although now I think its better to improve the constrained optimization before we move to multiple clusters :)
< sumedhghaisas>
hmmm
< sumedhghaisas>
okay give a trail run for 3 or 4 clusters and see whats happeneing
< KimSangYeon-DGU>
Ah, okay
< sumedhghaisas>
do you have any question regarding multiple cluster scenario?
< KimSangYeon-DGU>
About the data set,
< KimSangYeon-DGU>
Would it be a good idea to generate using GMM?
< KimSangYeon-DGU>
I mean classical GMM
< sumedhghaisas>
ohh that too. we still haven't looked at if QGMM is more powerful that GMM
< sumedhghaisas>
another research option :)
< KimSangYeon-DGU>
Wow
< KimSangYeon-DGU>
Good
< sumedhghaisas>
yeah for no you can generate using GMM and train QGMM on it
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
or just create 5 gaussains and sample from it
< sumedhghaisas>
thats much easier
< KimSangYeon-DGU>
Okay :)
< sumedhghaisas>
the point is to see how feasible is it
< sumedhghaisas>
just a trail run
< KimSangYeon-DGU>
Got it, I'll keep in mind
< sumedhghaisas>
lets also think about how can we setup experiments to prove QGMM is better than GMM
< sumedhghaisas>
maybe data some data in the middle of 2 clusters and see if the phase angle is changing accordingly
< sumedhghaisas>
the phase angle should change with the amount of data between the 2 clusters
< KimSangYeon-DGU>
Great, with 3D plot, we can see them more clearly
< sumedhghaisas>
yup
< sumedhghaisas>
lets think on this till we generate remaining results.
< sumedhghaisas>
I am sure we can come up with a good plan for this research direction as well
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
Great. :) Do you have any more updates? For next meeting feel free to set it up anytime later this week.
< KimSangYeon-DGU>
Okay :)
< KimSangYeon-DGU>
When I'm done with the trainable theta and high lambda, I'll ping you
< sumedhghaisas>
Coolio.
< sumedhghaisas>
Best of luck for the traineable variables
< KimSangYeon-DGU>
And then I'll work on the multiple clusters and theta angle researches.
< KimSangYeon-DGU>
Thanks!
< KimSangYeon-DGU>
Ah, firstly, I should check if the angle is correct when optimizing :)
k3nz0_ has quit [Quit: Leaving]
ImQ009 has joined #mlpack
KimSangYeon-DGU has quit [Remote host closed the connection]
KimSangYeon-DGU has joined #mlpack
vivekp has joined #mlpack
KimSangYeon-DGU has quit [Ping timeout: 260 seconds]
< lozhnikov>
rcurtin, zoq: Hello, I added some research on different implementations of Dictionary Encoding.