ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
abernauer has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
k3nz0_ has joined #mlpack
k3nz0__ has quit [Ping timeout: 246 seconds]
KimSangYeon-DGU has quit [Remote host closed the connection]
xiaohong has quit [Remote host closed the connection]
xiaohong has joined #mlpack
KimSangYeon-DGU has joined #mlpack
sumedhghaisas has joined #mlpack
< KimSangYeon-DGU>
Hi Ghaisas!
< KimSangYeon-DGU>
sumedhghaisas: Hey Ghaisas!
< sumedhghaisas>
Hey :) Hows it going?
< sumedhghaisas>
I was just going through the document
< KimSangYeon-DGU>
Currently working
< sumedhghaisas>
sorry couldn't find some time before
< KimSangYeon-DGU>
Ahh, no worries
< sumedhghaisas>
Hows the multiple case going?
< KimSangYeon-DGU>
I'm writing the code
< KimSangYeon-DGU>
When I'm done, I'll let you know
< KimSangYeon-DGU>
I think I need some time
< sumedhghaisas>
Surely. Nicely written conclusion :)
< KimSangYeon-DGU>
Thanks :)
< sumedhghaisas>
Do you discuss it further here?
< sumedhghaisas>
I saw a case where 2 clusters have converged on 1 cluster
< KimSangYeon-DGU>
Ah, actually, no. Something I wan to say was written in the document
< sumedhghaisas>
that seems normal enough
< KimSangYeon-DGU>
Ah, when I used low lambda, the 2 clusters have converged on 1 cluster
< sumedhghaisas>
interesting ... so initial phi matters a lot it seems
< KimSangYeon-DGU>
Yeah, initial phi matters
< sumedhghaisas>
so when you put initial value as 90 what is the final value of phi that you get?
< KimSangYeon-DGU>
Wait a moment
< sumedhghaisas>
If you could, could you also add the graph for changing phi over iterations? that ould be useful in analysis
< KimSangYeon-DGU>
Depending on the test case, test case 4 is 84.2564
< KimSangYeon-DGU>
In test case 5, phi is 89.18866.
< sumedhghaisas>
so phi is not changing much from 90
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
in the document its test case 1 right?
< sumedhghaisas>
this might be due to the fact that our dataset is from independent clusters
< sumedhghaisas>
maybe we need to change the dataset to change phi
< KimSangYeon-DGU>
In the test case 1, the last phi was 89.28725
< KimSangYeon-DGU>
Ah, I'll try
< KimSangYeon-DGU>
Yes, right, the phi wasn't changed largely
< sumedhghaisas>
okay I have an interesting experiment to add to this to support this hypothesis
< sumedhghaisas>
so in Figure 2 (c)
< KimSangYeon-DGU>
Oh, yeah
< sumedhghaisas>
you have set the initial phi as 180
< sumedhghaisas>
also report the final phi, but I suspect its not changed much
< sumedhghaisas>
phi 180 signifies that 2 clusters are subtracting each other at the intersection
< sumedhghaisas>
thats why they can occupy 1 dataset cluster together
< KimSangYeon-DGU>
The final phi was 177.96943931
xiaohong has quit [Ping timeout: 246 seconds]
< KimSangYeon-DGU>
Ah
< sumedhghaisas>
what was the final phi in Figure 2 (b)
< sumedhghaisas>
?
< KimSangYeon-DGU>
92.32387192
< sumedhghaisas>
very very interesting
< sumedhghaisas>
so when it was zero it finds the correct phi also
< sumedhghaisas>
okay so the new experiment is
< sumedhghaisas>
in Figure 2 (c) experiment put the initial clusters little farther from each other than they are right now
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
check for different initial distances from each other
< sumedhghaisas>
I suspect that is what causing this
< KimSangYeon-DGU>
Okay
< KimSangYeon-DGU>
I'll do that
< sumedhghaisas>
again do the saem (a) (b) (c) with different initial distances
< KimSangYeon-DGU>
Got it
< sumedhghaisas>
I bet you will find some distance when they actually find the correct clusters with phi 90
< KimSangYeon-DGU>
Ah, yes
< sumedhghaisas>
this turned out to be a very good research direction as it also provides us clues for further research
< KimSangYeon-DGU>
Sounds good :)
< sumedhghaisas>
one viable nex direction is to change the dataset in appropriate way to see if its able to find different phis
< KimSangYeon-DGU>
Yes
< sumedhghaisas>
I have an idea in that way
< KimSangYeon-DGU>
What idea??
< KimSangYeon-DGU>
Can you tell me?
< sumedhghaisas>
In Figure 2 (b) you have found correct values
< KimSangYeon-DGU>
yeah
< sumedhghaisas>
so we have QGMM distribution in the end
< KimSangYeon-DGU>
Right
< sumedhghaisas>
use that distribution and change its phi value manually
< KimSangYeon-DGU>
Ah~
< KimSangYeon-DGU>
Nice idea
< sumedhghaisas>
make it lets say 120 or 30
< sumedhghaisas>
and then normalize it again
< sumedhghaisas>
and sample from it
< sumedhghaisas>
hmmm... but how would you sample from QGMM that we need to understand
< sumedhghaisas>
we only have PDF and CDF cannot be computed
< KimSangYeon-DGU>
Hmm..
< sumedhghaisas>
we have to use some crazy sampling technique for this.. simple CDF technique won't cu it
< sumedhghaisas>
That might be ou next research topic
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
I'll think about it
< sumedhghaisas>
how to sample from QGMM ... if we could do that we can create a good dataset
< sumedhghaisas>
wait there another hacky way of doing it
< sumedhghaisas>
so when you generate the normal dataset
< sumedhghaisas>
create a circle in the middle of the dataset ... middle of the 2 centers
< sumedhghaisas>
with radius R
< KimSangYeon-DGU>
Yeah
< sumedhghaisas>
and delete 67 percent points in the circle
< sumedhghaisas>
change R to get different datasets
< sumedhghaisas>
67 percent would basically generate Gaussian like effect
< KimSangYeon-DGU>
Ahh..
< sumedhghaisas>
we can test Figure 2 (b) situation for these different datasets to see what phi do we get
< sumedhghaisas>
if we are still getting 90 then something is wrong
< KimSangYeon-DGU>
Yeah, thanks for the idea
< KimSangYeon-DGU>
If I'm get stuck in doing that research, can I ask you questions later?
< sumedhghaisas>
Surely. Anytime.
< sumedhghaisas>
I always like some good research
< KimSangYeon-DGU>
:)
< sumedhghaisas>
Try to generate the datasets first and see if they look different enough
< KimSangYeon-DGU>
Got it, your idea is really nice
< sumedhghaisas>
rest of the job is straightforward
< KimSangYeon-DGU>
Generating the dataset
< sumedhghaisas>
lets hope it works
< KimSangYeon-DGU>
Yeah
< KimSangYeon-DGU>
Thanks!
< sumedhghaisas>
if the phi is not changing much in the research document you sent me
< sumedhghaisas>
just note down the final phi values as well
< sumedhghaisas>
no need to add graphs
< sumedhghaisas>
little less work I guess
< sumedhghaisas>
Okay so far we seem to have 3 tasks at hand.
< sumedhghaisas>
1. Change distance in Figure 2 (c) and see the effect
< sumedhghaisas>
2. Multiple cluster case
< sumedhghaisas>
3. crazy gaussian effect idea with radius change
< KimSangYeon-DGU>
Right :)
< sumedhghaisas>
I will leave it to you to prioritize
< KimSangYeon-DGU>
Ah, thanks
< sumedhghaisas>
and take your time ... its always better to get done the right way :)
< KimSangYeon-DGU>
I agree :)
< sumedhghaisas>
Do you have any questions in the multiple cluster case?
< KimSangYeon-DGU>
Thanks for organizing the tasks
< KimSangYeon-DGU>
How should I manage Phi?
< KimSangYeon-DGU>
as a matrix?
< sumedhghaisas>
good question. but it won't be between 2 clusters anymore right
< sumedhghaisas>
it will be assigned to each cluster separately?
< sumedhghaisas>
just want make sure I understand correctly
< KimSangYeon-DGU>
Hmm
< KimSangYeon-DGU>
Can you give some time to think about it?
< sumedhghaisas>
basically it will be equation 6 in the paper right?
< sumedhghaisas>
where each cluster 'k' has phi_k
< KimSangYeon-DGU>
Ah, yeah, but it is equation (7)?
< KimSangYeon-DGU>
Ahh, I understand
< KimSangYeon-DGU>
Because cos(phi)_{1,2} = cos(phi)_{2,1}, is it possible to assign the phi to each cluster separately?
< KimSangYeon-DGU>
Actually, I intended to use a matrix
< sumedhghaisas>
ahh yes equation 7 my bad
< sumedhghaisas>
yes just create that many traineable variables
< sumedhghaisas>
basically whenever you are using equation 7 in the code
< sumedhghaisas>
use all the variables to generate the subtractions
< sumedhghaisas>
matrix will be little harder to implement
< sumedhghaisas>
this will be easier as its just subtractions basically... in a loop
< KimSangYeon-DGU>
Okay
< sumedhghaisas>
And yes take your time to understand
< KimSangYeon-DGU>
Yeah :)
< sumedhghaisas>
you can setup something again this week to ask questions
< KimSangYeon-DGU>
Cool!
< KimSangYeon-DGU>
Thanks for the meeting
< KimSangYeon-DGU>
I'll ping you
< sumedhghaisas>
while you are trying to understand this I would reccommend prioritizing other 2 task while doing this
< KimSangYeon-DGU>
Ahh, right
< sumedhghaisas>
lets take our time to implement multiple cluster case as its little bit complex
< KimSangYeon-DGU>
Okay
< sumedhghaisas>
I need to attend to another meeting now if you still have some questions could you send an email or ping me on Hnagout?
< KimSangYeon-DGU>
No :)
< KimSangYeon-DGU>
Yeah, If I have a question, I'll ping you or send emails :)
< sumedhghaisas>
great. Gotta run. Best of luck for the work
< KimSangYeon-DGU>
Thanks!!
k3nz0__ has joined #mlpack
k3nz0_ has quit [Ping timeout: 248 seconds]
xiaohong has joined #mlpack
ImQ009 has joined #mlpack
xiaohong has quit [Remote host closed the connection]
KimSangYeon-DGU has quit [Remote host closed the connection]
gtank___ has quit [Ping timeout: 244 seconds]
gtank___ has joined #mlpack
gtank___ has quit [Ping timeout: 252 seconds]
gtank___ has joined #mlpack
sumedhghaisas has quit [Ping timeout: 260 seconds]
vivekp has quit [Ping timeout: 245 seconds]
< sreenik[m]>
zoq: Thanks for the comment on the PR. I will sort that out. On a different note, I thought of asking you one thing. For the *mlpack-onnx* converter, it would be helpful to have an ONNX API that builds a model layer by layer (just like FFN in mlpack or the somewhat roundabout procedure in Torch). However, none exists at the moment for C++. Now, the onnx model being a protobuf file it is possible to create a valid model
< sreenik[m]>
from scratch but it will be rather painstaking to do so. So as to avoid this, I am thinking of creating a *mlpack-Torch* converter instead. The reason I chose Torch (over Tensorflow, etc.) is that Torch itself maintains a Torch to ONNX converter (in contrary to ONNX maintaing it), not to mention that it has a solid documentation too. However, Torch brings in a dependency of about 180MB. What would you suggest?
< zoq>
sreenik[m]: So this means I could convert from mlpack to torch and from torch to whatever is supported?
< sreenik[m]>
Yes exactly. Secondly as torch has a torch to onnx converter, we can convert to ONNX and then to anything we want\
ImQ009 has quit [Quit: Leaving]
< zoq>
sreenik[m]: I see, so the base for us isn't ONNX but torch. I guess the only issue is that torch does come with a bigger footprint (180MB)?
< sreenik[m]>
That's the only issue, hopefully
< zoq>
sreenik[m]: Personally I don't mind to go via torch, if it makes things easier or even opens more opportunities.
< zoq>
sreenik[m]: If Atharva and you like the idea, fine with me; probably a good idea to check the pipline before.
< sreenik[m]>
Yes Atharva and I have discussed this issue. We wanted to take your opinion before proceeding. So would you suggest to create a mini-converter (with one or two supported layers) and see if everything is going right?
< zoq>
sreenik[m]: Yes, I think that is a reasonable approach, might give us some insight about what works and what not.
< sreenik[m]>
Yup. I'd need to look into extracting the layers from the mlpack model too. I hope that won't be a hassle
< sreenik[m]>
zoq: Do you remember if mlpack has biases stored before weights in the *parameters* matrix?
< zoq>
sreenik[m]: Maybe you can go even simpler with LinearNoBias instead of Linear.
< zoq>
sreenik[m]: The bias comes last.
< zoq>
sreenik[m]: Maybe LinearNoBias followed by Sigmoid?
< sreenik[m]>
That too sounds reasonable. Quite relieved it's in the last, I have assumed it throughout in the onnx-mlpack converter that's completed
< zoq>
sreenik[m]: Good, would be easy to change the position.