ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
KimSangYeon-DGU has quit [Remote host closed the connection]
vivekp has joined #mlpack
< ShikharJ>
sakshamB: Toshal: I don't think I'll be able to make it to the meeting today. Let's talk on Monday? If you guys have a message, please feel free to leave them in the channel.
< sakshamB>
ShikharJ alright thats fine with me. Have a great weekend :)
KimSangYeon-DGU has joined #mlpack
sumedhghaisas has joined #mlpack
< sumedhghaisas>
KimSangYeon-DGU: Hey
< sumedhghaisas>
Sorry for the delay
< KimSangYeon-DGU>
Hi Sumedh!!
< KimSangYeon-DGU>
No worries :)
favre49 has joined #mlpack
< KimSangYeon-DGU>
I implemented QGMM and found the method to update alpha using the NLL + lambda * approx constant
< KimSangYeon-DGU>
Lagrange multiplier
< sumedhghaisas>
great. So what lambda value did you use?
< KimSangYeon-DGU>
I'll update the equation for alpha optimization soon
< KimSangYeon-DGU>
Ah, I treat the lambda as constant
< KimSangYeon-DGU>
It cancelled out later
< sumedhghaisas>
hmmm... okay little confused
< KimSangYeon-DGU>
When I implemented QGMM, it seem to be main point to optimize the alpha
< sumedhghaisas>
so you implemented NLL + lambda * approx constant
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
I used the equation to calculate the alpha
< sumedhghaisas>
and whats the update procedure?
< KimSangYeon-DGU>
First,
< KimSangYeon-DGU>
I add KMeans clustering to initialize the parameters
< KimSangYeon-DGU>
Second, implemented QGMM except for alpha optimization
< KimSangYeon-DGU>
Third, I calculated the alpha optimization
< KimSangYeon-DGU>
Oops
< KimSangYeon-DGU>
The second progress is several days ago
< sumedhghaisas>
what do you mean QGMM except for alpha optimzation?
< KimSangYeon-DGU>
The third progress is recent progress
< KimSangYeon-DGU>
Ahh,
< KimSangYeon-DGU>
At first, when I implemented the QGMM, I didn't know how I can update the alpha
< KimSangYeon-DGU>
So, I thought about the equation you said
< KimSangYeon-DGU>
NLL + lambda * approx const
< sumedhghaisas>
huh
< sumedhghaisas>
so you update which parameters with QGMM?
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
finally, I updated the alpha, theta
< KimSangYeon-DGU>
but alpha optimization needs to be verified
< KimSangYeon-DGU>
Because, I just calculated it
< KimSangYeon-DGU>
I wanted to check the equations to you
< KimSangYeon-DGU>
*want
< sumedhghaisas>
wait... still confused. So you initialized all the parameters
< sumedhghaisas>
there are means variance thera and alpha
< sumedhghaisas>
so you used the equation given in the paper?
< KimSangYeon-DGU>
I initialized all the parameters using K Means clustering algorithm and then trained the parameter using the equations in my final proposal
< sumedhghaisas>
okay ... all the parameters?
< KimSangYeon-DGU>
Ahh
< sumedhghaisas>
I mean trained all the parameters using the equations?
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
so where is NLL + lambda * constraint is used?
< KimSangYeon-DGU>
for the alpha optimization
< sumedhghaisas>
hmmm ... okay I think code would make me understand it little better :P
< KimSangYeon-DGU>
Oh.. sorry for confusing...
< sumedhghaisas>
So mean variance and theta is updated using the equations in your paper
< sumedhghaisas>
but for alpha you use NLL equation?
< KimSangYeon-DGU>
Right
< sumedhghaisas>
I see.
< sumedhghaisas>
so in iteration of the update its first update mean variance and theta
< sumedhghaisas>
and then use NLL equation to update alpha...
< sumedhghaisas>
hmm... okay firstly is this procedure converging?
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
interesting...
< sumedhghaisas>
okay going through the code again in some details
< KimSangYeon-DGU>
Thanks!!
< sumedhghaisas>
Just a quic question... in the code which is the alpha parameter
< sumedhghaisas>
?
< KimSangYeon-DGU>
Ah, it is named weights
< KimSangYeon-DGU>
I'll change it
< KimSangYeon-DGU>
in the QuantumGMM class
< sumedhghaisas>
ahh okay that make sense
< KimSangYeon-DGU>
QuantumGMM has alpha parameter as 'weights' variable
< sumedhghaisas>
also why are you using the KMeans fit?
< KimSangYeon-DGU>
Ahh, because the classical EM algorithm uses KMeans to initialize the parameters to train.
< sumedhghaisas>
umm... it should also work without. Did you try without?
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
For simple enough dataset it should find the centers
< KimSangYeon-DGU>
If we don't use it, It's not trained well
< KimSangYeon-DGU>
When I introduce the KMeans to QGMM, the performance increases
< sumedhghaisas>
okay. But did you see the means and variance after KMeans finishes?
< sumedhghaisas>
maybe the result is already really close?
< KimSangYeon-DGU>
I checked it
< KimSangYeon-DGU>
It is not close
< sumedhghaisas>
interesting...
< KimSangYeon-DGU>
You can check the performance by typing "python main.py"
< sumedhghaisas>
okay could you give me the means before and after QGMM algorithm?
< KimSangYeon-DGU>
Yes
< sumedhghaisas>
ahh okay. does it print just after KMeans too
< KimSangYeon-DGU>
Wait a moment
< sumedhghaisas>
I am trying to run but its missing some sklearn.cluster
< KimSangYeon-DGU>
Initial mean is zero
< sumedhghaisas>
I will install it later and run the file
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
Sumedh
< sumedhghaisas>
Don't want to keep you awake for too long actually
< KimSangYeon-DGU>
The covariance is quite a between KMeans and QGMM
< KimSangYeon-DGU>
but mean is quite similar
< KimSangYeon-DGU>
sorry for the confusing but QGMM is more accurate
< KimSangYeon-DGU>
I'm okay :)
< KimSangYeon-DGU>
I used the two distributions, representing each class
< KimSangYeon-DGU>
initial means of d1 and d2 is zero
< KimSangYeon-DGU>
After KMeans, the mean of d1 is [ 5.15701407 6.36799727 3.09944505 3.18260357 2.03883024]
< KimSangYeon-DGU>
the mean of d1 is [ 0.96782953 -0.9264934 -0.01378471 1.00829962 0.89609672]
< KimSangYeon-DGU>
After QGMM, the mean of d1 is [ 5.01880026 6.16773633 2.96954073 3.10592078 1.93759082]
< KimSangYeon-DGU>
the mean of d2 is [ 0.93731248 -1.00229649 -0.02631165 0.9955708 0.91626373]
< KimSangYeon-DGU>
Finally, the actual mean of d1 is [5, 6, 3, 3, 2]
< KimSangYeon-DGU>
the actual mean of d2 is [1, -1, 0, 1, 1]
< sumedhghaisas>
hmmm... the means are really close I think
< sumedhghaisas>
could you try with initialization little more far away?
< KimSangYeon-DGU>
Yeah, the covariance is really different
< sumedhghaisas>
maybe our method converges when the parameters are correct?
< KimSangYeon-DGU>
Hmm..
< sumedhghaisas>
that also
< sumedhghaisas>
Also I thought NLL equation will be used to update ll the parameters :)
< sumedhghaisas>
basically you take gradient with respect to each one
< sumedhghaisas>
I have a suspicion that the update equation are wrong...
< KimSangYeon-DGU>
Oops... which one??
< sumedhghaisas>
ahh no not the code
< sumedhghaisas>
so the formulation they use which is
< KimSangYeon-DGU>
Ahh
< sumedhghaisas>
so you take equation 16 and differentiate it
< sumedhghaisas>
with each parameter right?
< KimSangYeon-DGU>
Yeah, right
< sumedhghaisas>
but in the derivation the Q is expanded right?
< sumedhghaisas>
which I think is wrong. Because Q is just an estimate using the last iteration parameters
< sumedhghaisas>
parameters inside Q are not variables but constants
< sumedhghaisas>
For example
< sumedhghaisas>
if you see GM update you do E step to do estimates which are used in M step as constants
< sumedhghaisas>
If you see derivation of GMM EM update you will see the same behavior
< sumedhghaisas>
Q is taken as an estimate and not an equation
< KimSangYeon-DGU>
Ahh...
< sumedhghaisas>
E step and M step are separate in that way
< KimSangYeon-DGU>
So, I should re-derivate
< KimSangYeon-DGU>
Oops...
< sumedhghaisas>
but if you don't expand the Q you don't get any closed form solution
< sumedhghaisas>
I tried that
< KimSangYeon-DGU>
Hmm...
< KimSangYeon-DGU>
When I used equations
< KimSangYeon-DGU>
the covariances are trained well
< KimSangYeon-DGU>
but the KMeans didn't
< sumedhghaisas>
Now at the minimum point the estimate the update should give the same answers
< sumedhghaisas>
but then the complete formed solution which we don't get
< sumedhghaisas>
also in your derivation
< sumedhghaisas>
i mean their derivation as well
< sumedhghaisas>
when they expand the Q
< sumedhghaisas>
equation 33
< sumedhghaisas>
there are G's as well in the equation
< sumedhghaisas>
nor G also contains variable 'mu' which is the mean
< sumedhghaisas>
now why is that variable treated different that othr 'mu' variables?
< sumedhghaisas>
this is all very non mathy
< KimSangYeon-DGU>
Agreed
< sumedhghaisas>
I have never seen an EM proof that actually does this kinda thing
< sumedhghaisas>
actually no EM proof ever expands Q
< sumedhghaisas>
cause that is a constant
< sumedhghaisas>
Actually you should read the derivation of GMM EM
< sumedhghaisas>
maybe you will understand better
< KimSangYeon-DGU>
Yeah, I understand
< sumedhghaisas>
So thats why we were just going to bail on the updates
< sumedhghaisas>
we use good old gradient descent
< sumedhghaisas>
use NLL + lambda * constraint to do gradient descent
< sumedhghaisas>
but the its good to know that the updates converge around correct answer
< KimSangYeon-DGU>
for alpha, mean, and covariances?
< sumedhghaisas>
actually that makes sense
< sumedhghaisas>
near the correct answer
< sumedhghaisas>
estimate and the updated parameters will be same
< sumedhghaisas>
so Q can be taken as constant or as equation
< sumedhghaisas>
hmmm
< sumedhghaisas>
we need to think ths through some more
< sumedhghaisas>
ahh yes
< sumedhghaisas>
all parameters could be updated with gradient descend
< KimSangYeon-DGU>
I have a question
< sumedhghaisas>
okay
< KimSangYeon-DGU>
In the unsupervised learning, can we use gradient descent?
< sumedhghaisas>
yes surely
< KimSangYeon-DGU>
Ahh
< KimSangYeon-DGU>
Thanks
< sumedhghaisas>
we use gradient descend to maximize LL
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
or to minimize NLL
< sumedhghaisas>
given constarint that becomes NLL + lambda * constraint
< sumedhghaisas>
so its really easy
< sumedhghaisas>
loss is that equation
< sumedhghaisas>
use Tensorflow or pytorch to update all parameters using Adam optimizer or something
< KimSangYeon-DGU>
Yeah, I'll try it
< sumedhghaisas>
I don't have strong confidence it will work but we can try
< KimSangYeon-DGU>
Right
< sumedhghaisas>
theoretically it should work with correct constraint
< sumedhghaisas>
but we are using approximate constraint remember?
< KimSangYeon-DGU>
Can you remind me of it?
< sumedhghaisas>
so the constraint is given just under equation 9
< KimSangYeon-DGU>
Thanks
< sumedhghaisas>
that the probability distribution should integrate to
< sumedhghaisas>
1
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
but we can find that integral
< sumedhghaisas>
so we just use the current batch of points to estimate it
< KimSangYeon-DGU>
Ah, for normalizing?
< sumedhghaisas>
yes yes
< KimSangYeon-DGU>
I remember we use the batch for normalization
< KimSangYeon-DGU>
ah Cool
< sumedhghaisas>
so we do NLL + lambda * constraint for a batch
< sumedhghaisas>
and optimize it
< sumedhghaisas>
i am not sure it works but lets try :P
< KimSangYeon-DGU>
Got it :)
< sumedhghaisas>
okay you should sleep now.
< KimSangYeon-DGU>
Yeah, thanks for the meeting!
< sumedhghaisas>
we are already way past 2AM :P
< KimSangYeon-DGU>
:)
< sumedhghaisas>
lets catch up tomorrow over hangouts if you have any questions
< KimSangYeon-DGU>
Okay!
< sumedhghaisas>
I will be out tomorrow so won't be able to get table connection for IRC
< favre49>
zoq: In your experience, how long does NEAT take on double pole balancing with no velocities?
< KimSangYeon-DGU>
Ah~ that make sense
< sumedhghaisas>
*stable
< KimSangYeon-DGU>
Thanks again :)
< favre49>
I tested it on double pole balancing with velocities, and the results are almost too good, it seems the initial population itself usually has a member that can last indefinitely (probably through wiggling)
< favre49>
On double pole balancing with velocities, I have run it for 500-700 generations, and the agent usually reaches 450ish steps.
< favre49>
I'm making some changes to the fitness function though, so hopefully it gets better with that and some parameter tweaking
< favre49>
Also, I was wondering what other features we should add to the NEAT implementation once testing is done (hopefully soon).
< favre49>
In my proposal I mentioned Phased Searching from SharpNEAT which claimed to reduce genome bloat, but I'm not sure how much of a problem that is, based on the testing I've done. Moreover, the author argued that speciation prevents it. I'm ambivalent though, it may be interesting to try. I'll do what you say on that one.
favre49 has quit [Remote host closed the connection]
< zoq>
favre49: Don't really have a number in mind, but around 500 sounds reasonable. In your tests you are going for a single run, but note that e.g. the CartPole task is "considered "solved" when the agent obtains an average reward of at least 195.0 over 100 consecutive episodes". We should run consecutive episodes as well.
< zoq>
favre49: I really liked the idea (Phased Searching), so I would test it out.
< zoq>
favre49: Maybe it makes sense to run NEAT against OpenAI's gym, that way we could get some recordings and see what the result looks like.
< zoq>
favre49: If you like I can set something up.
favre49 has joined #mlpack
< favre49>
zoq Sounds good, the OpenAI Gym idea also sounds great.
< favre49>
What do you think about implementing some sort of config file?
< zoq>
favre49: Right, definitely a good idea, perhaps the mlpack serialization feature is all we need, we could write a simple class that holds some parameter and see if the serialization output (txt, xml) looks simple enough. What do you think?
< favre49>
zoq Yup, are there any examples I can read for this?