verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
mrcode has joined #mlpack
rajeshdm9 has joined #mlpack
csoni has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
rajeshdm9 has quit [Ping timeout: 260 seconds]
ravikiran0606 has joined #mlpack
ravikiran0606 has quit [Client Quit]
csoni has joined #mlpack
mrcode has quit [Quit: Leaving.]
MystikNinja has joined #mlpack
MystikNinja has quit [Client Quit]
csoni2 has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
rajeshdm9 has joined #mlpack
ironstark_ has quit [Quit: Connection closed for inactivity]
csoni has joined #mlpack
csoni2 has quit [Read error: Connection reset by peer]
csoni has quit [Ping timeout: 268 seconds]
rajeshdm9 has quit [Ping timeout: 260 seconds]
csoni has joined #mlpack
csoni has quit [Ping timeout: 240 seconds]
csoni has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
donjin_master has joined #mlpack
< donjin_master>
Hello everyone, i want to submit my proposal on reinforcement learning . Can anyone suggest me what should i have to write in the proposal
< donjin_master>
Like i want to work on double DQN Algorithm does i have to explain that how will i am going to implement this algorithm in summer and the proper timelone of the work schedule
< donjin_master>
Correct me if a i am wrong ...
< donjin_master>
Will i have to give the proper code in my proposal for the algorithm
< donjin_master>
Thanks zoq i have gone through the application guide i am going to draft a proposal as soon as possible ..
donjin_master has quit [Client Quit]
csoni has joined #mlpack
haritha1313 has joined #mlpack
< haritha1313>
@rcurtin: @zoq: I am writing my proposal based on neural collaborative filtering. And as per discussions in mailing list, I tried to benchmark it against existing cf algorithms in mlpack.
< haritha1313>
Since the implementation available for ncf uses hit ratio and ndcg as parameter, and mlpack uses rmse, I calculated rmse for ncf. On comparison ncf is giving 1.6 (approx) rmse and most algorithms in mlpack are giving rmse greater than 2.
csoni has quit [Read error: Connection reset by peer]
< haritha1313>
Do you think these results are enough to check whether it is worth implementing? Or should I calculate hit ratio in mlpack's cf for comparison?
< zoq>
haritha1313: I think this is just fine.
< haritha1313>
@zoq: Thanks :)
< zoq>
Thanks for taking the time to do the comparison.
haritha1313 has left #mlpack []
prem_ktiw has joined #mlpack
csoni has joined #mlpack
sujith_ has joined #mlpack
csoni2 has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
csoni has joined #mlpack
csoni2 has quit [Read error: Connection reset by peer]
satyam_2401 has joined #mlpack
rf_sust2018 has joined #mlpack
prem_ktiw has quit [Ping timeout: 245 seconds]
sujith_ has quit [Ping timeout: 260 seconds]
rf_sust2018 has quit [Quit: Leaving.]
rf_sust2018 has joined #mlpack
csoni has quit [Ping timeout: 264 seconds]
csoni has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
ImQ009 has joined #mlpack
IAR has joined #mlpack
IAR has quit [Client Quit]
IAR_ has joined #mlpack
IAR has joined #mlpack
csoni has joined #mlpack
IAR_ has quit [Client Quit]
IAR has quit [Read error: Connection reset by peer]
satyam_2401 has quit [Quit: Connection closed for inactivity]
yashsharan has joined #mlpack
< yashsharan>
@zoq Did you get the chance to review my draft?thanks.
csoni has quit [Read error: Connection reset by peer]
csoni has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
Prabhat-IIT has joined #mlpack
< Prabhat-IIT>
zoq: Please look into the SAGA optimizer pr as its on the verge of ready if we can handle the random seed issue :)
< Prabhat-IIT>
zoq: Btw what do you think about the topology of particle in PSO that I've elaborated in my draft?
donjin_master has joined #mlpack
csoni has joined #mlpack
< donjin_master>
Can someone suggest me how can i implement reinforcement learning algorithm in my machine with using cartpole and mountain car environment
donjin_master has quit [Client Quit]
csoni has quit [Ping timeout: 264 seconds]
sumedhghaisas has joined #mlpack
Abkb has joined #mlpack
Abkb has quit [Client Quit]
< Atharva>
sumedhghaisas: the way mlpack optimizer object works is that it expects the loss function as the last layer of the network object. In variational autoencoders, the loss function is the sum of kl divergence and reconstruction loss
< Atharva>
Which usually is negative log likelihood or mean squared
rf_sust2018 has quit [Quit: Leaving.]
< Atharva>
Should I propose to define a new layer/loss function including both or should i try to find out a way to combine two cost/layer objects?
csoni has joined #mlpack
sourabhvarshney1 has joined #mlpack
rf_sust2018 has joined #mlpack
< sumedhghaisas>
@Atharva: Hi Atharva
< sumedhghaisas>
Let me see if I understand your question correctly.
sumedhghaisas2 has joined #mlpack
sumedhghaisas has quit [Read error: Connection reset by peer]
< sumedhghaisas2>
@Atharva: sorry for that... connection problem
< Atharva>
No problem
< sumedhghaisas2>
okay so in the current class FFN and RNN the loss function can only be dependent on the last layer output
< sumedhghaisas2>
although KL divergence will be dependent on the middle layer Z ... is that correct?
csoni has quit [Read error: Connection reset by peer]
< Atharva>
Yeah, that’s correct
< sumedhghaisas2>
okay. Yes, I have considered this problem while proposing VAE framework.
< Atharva>
when we call the optimize function from the train function, we just pass in the entire network as the first argument, for VAEs, we need to optimize the parameters wrt to the combined loss function
< sumedhghaisas2>
if we want to make it work with current framework there are couple of options
< sumedhghaisas2>
but all of them would involve changing lot of pre-existing code.
< Atharva>
Okay, what are the options?
< sumedhghaisas2>
hmmm... although the same problem will be faced by GANs, there are 2 networks to optimize
< sumedhghaisas2>
we have to look in their code to figure out the way they are optimizing 2 networks. The same will apply for our Encoder and Decoder. what you think?
< Atharva>
Sorry, I am not quite sure about how GANs are trained, but both the encoder and decoder of the VAE are trained as one network, correct me if I am wrong
< sumedhghaisas2>
Yes you are right. But the problem we are facing might be similar to the problem faced in GAN. let me see if I can explain this properly.
< sumedhghaisas2>
Ahh but that is only if the GANs implementation in MLPACK is using variational approximations. I see.
< sumedhghaisas2>
Sorry :) I am not sure about the GAN implementation in MLPACK either.
< sumedhghaisas2>
so the options...
< sumedhghaisas2>
One preferred would be to implement a separate class which supports training with variationql approximations, technique which involved KL in the loss.
< sumedhghaisas2>
*involves
< sumedhghaisas2>
Another is to change the current framework such that the loss is dependent on each layer. Some sort of visitor which takes loss term from each layer.
< Atharva>
Do you mean that we will have a class which will already have the KL loss and then to his class we will also pass the reconstruction loss, then this combined cost will be optimized.
< Atharva>
The above doubt is for the first option
< sumedhghaisas2>
umm. Not really.
< sumedhghaisas2>
so FFN and RNN both do not support Variational loss, which may defined over middle layer.
< sumedhghaisas2>
so the first option involves implementing a new framework altogether which handles Variational losses better.
< Atharva>
Oh
< Atharva>
But that doesn’t seem to be a good solution just to fit in a type of loss
< sumedhghaisas2>
I agree. So the second option...
< Atharva>
The second option, instead of a visitor that takes loss from every layer(which we don’t need) can’t we maintain a pointer to the middle layer and take loss only from this
< sumedhghaisas2>
Yes we can. but where will you maintain a pointer? I mean how will you tell FFN class to maintain such a pointer?
< sumedhghaisas2>
The problem with FFN class is that it treats all the layers equally.
< Atharva>
So what I am planning to do is, instead of having the Variational autoencoder as one FFN object, I will have the encoder as one, the decoder as one, and the middle layer(that samples latent variables) as a different layer object altogether
< Atharva>
These objects will be inside the VAE class which i will define
< sumedhghaisas2>
Ahh yes. Basically that's the first option. :)
< sumedhghaisas2>
So the newly created class will become parallel to FFN and RNN
< sumedhghaisas2>
this way if other frameworks use vational inference, they will use this newly created framework for training....
< Atharva>
Oh, okay
< Atharva>
Yeah, okay, so in order to train this network, we will meed to modify a lot of code
< sumedhghaisas2>
The first option involves lot of work but in my opinion is long lasting. Although we should talk to @zoq and @rcurtin about this before proceeding. They might have a better idea about this.
< sumedhghaisas2>
various GAN uses Variational generator to speed up the training or that's what I have heard
< sumedhghaisas2>
there this class can be used.
< Atharva>
Sorry I didn’t get the first option when you were explaining it.
< sumedhghaisas2>
zoq will know more this than me.
< sumedhghaisas2>
don't worry :) I am not very good at explaining either.
< sumedhghaisas2>
let's be happy that now we are on same page
< Atharva>
Yeah :)
< Atharva>
I do think this is a really good option
< sumedhghaisas2>
Although we must be careful to make it as generic as possible. Lot of genertive models are hierarchical
< Atharva>
Because when using VAEs, we should have the freedom to use the encoder and decoder independently, especially for advanced users
< sumedhghaisas2>
as in they may have hierarchical latents in them
< Atharva>
Yes I understand, it needs to be very generic
< sumedhghaisas2>
thus the loss might be dependent on various KL divergences
< Atharva>
Okay, do you mean there should be a way to keep track of multiple sampling layers which can have KL divergence
< sumedhghaisas2>
Also I am not sure that the current FFN supports using distributions as output
< sumedhghaisas2>
@Atharva: yes precisely.
< Atharva>
About that, the latent variable sampling layer which i plan to implement will take care of the reparametrization trick used to train VAEs
< Atharva>
I mean..
< sumedhghaisas2>
We need not implement such a hierarchical VAE, we need to make sure someone who wants to can use our API without much of changes.
< Atharva>
Sorry I didn’t quite get that
< sumedhghaisas2>
I mean we should design the class such that there can be multiple sampling layers
< Atharva>
I meant that the encoder will output means standard deviations and pass them to the sampling layer, this layer will then sample a point from standard normal. By multiplying this by the variance and adding mean we have a sample
< Atharva>
I think this layer could be used anywhere even multiple times within the network
< sumedhghaisas2>
Yes that should work. Also the final loss shoukd be defined over KL of all these layers.
< Atharva>
We can maintain pointers to all such layers we have in the network and take KL divergences from them
< Atharva>
Exactly
< Atharva>
So, is this all a good plan
< sumedhghaisas2>
We should still think about this and try to come up with different options if possible, before the coding actually begins.
< sumedhghaisas2>
to make sure we are not missing anything
< Atharva>
Yeah, but only 7 days remain for the proposal deadline, what do you think I should put as a plan there?
< sumedhghaisas2>
discussion it with zoq and rcurtin will also give us some more ideas and perspectives
< sumedhghaisas2>
as the API is tentative, mentioning the gist of the discussion is also an option. :)
< Atharva>
Also, can you point me to some paper/blog post with models that require multiple sampling layers
< Atharva>
Okay, i will do that, thank you! :)
< sumedhghaisas2>
PixelVAE uses PixelCNN layer as decoder in VAE.
< sumedhghaisas2>
for cifar they train a hierarchical model indeed.
< Atharva>
Oh, okay, so I guess it’s really important to make it as generic as possible so that every possible VAE model can be made using the framework
sumedhghaisas2 has quit [Read error: Connection reset by peer]
sumedhghaisas has joined #mlpack
< Atharva>
zoq:
< Atharva>
rcurtin:
< Atharva>
It will be very helpful if you could give your opinions regarding this discussion. Please do when you get the time.
< sumedhghaisas>
@Atharva: Yes. I am just starting with a meeting so might be delayed to respond, I will try to reply after the meeting.
< Atharva>
Yeah no problem, i will work on what we have discussed till now
csoni has joined #mlpack
csoni has quit [Read error: Connection reset by peer]
IAR has joined #mlpack
IAR has quit [Quit: Leaving...]
sumedhghaisas has quit [Read error: Connection reset by peer]
sumedhghaisas has joined #mlpack
sumedhghaisas has quit [Ping timeout: 256 seconds]
sumedhghaisas has joined #mlpack
sumedhghaisas has quit [Ping timeout: 256 seconds]
sourabhvarshney1 has quit [Ping timeout: 260 seconds]
sumedhghaisas has joined #mlpack
sumedhghaisas2 has joined #mlpack
sumedhghaisas has quit [Read error: Connection reset by peer]
sumedhghaisas2 has quit [Read error: Connection reset by peer]
sumedhghaisas has joined #mlpack
rf_sust2018 has quit [Quit: Leaving.]
sumedhghaisas has quit [Read error: Connection reset by peer]
sumedhghaisas has joined #mlpack
sumedhghaisas has quit [Ping timeout: 248 seconds]
sumedhghaisas has joined #mlpack
yashsharan has quit [Ping timeout: 260 seconds]
ImQ009 has quit [Read error: Connection reset by peer]
sumedhghaisas has quit [Read error: Connection reset by peer]