03:05 < Ankit> Hello people

03:08 < Ankit> I am an CS under grad and a beginner in the field of Machine learning .I am interested in contributing . So can you please guide me and tell me the prerequisites.It will e a great help and I will be grateful to you all

05:28 < lozhnikov> kris1: that's ok. the error doesn't go down since that's a minimax game. If you apply the patch as is you should get the following results https://usercontent.irccloud-cdn.com/file/ZiGRmGjp/results.png

05:28 < lozhnikov> kris1: I prefer a separate PR

07:23 < partobs-mdp> zoq: rcurtin: Implemented Parameters() method as rcurtin described (make individual function parameters as memory pointers to some contiguous memory block), but there are still only zeros in the HAMUnit paramters. Can you take a look at the issue?

08:36 < kris1> Hi, Lozhnikov

08:36 < kris1> I will send you the files that i have for GAN PR and also the gan.cpp file.

08:37 < kris1> I don’t get the results that you mentioned.

08:37 < kris1> Also i was able to create ResizeLayer. There is some error regarding layer types i still have to figure that out.

08:37 < kris1> Here is my branch for ResizeLayer.

08:42 < kris__> https://usercontent.irccloud-cdn.com/file/pYouJLZa/a.patch

08:43 < kris__> https://usercontent.irccloud-cdn.com/file/KNCXPfim/gan.cpp

08:43 < kris__> Could you send me dataset somehow...

08:45 < lozhnikov> Kris1: that's a dataset from the ssRBM PR. Digits_train.arm

08:48 < kris1> Hmmmm welll in that case i am also using the same data set.

08:49 < kris1> My time to convergence is around 16 min

08:50 < lozhnikov> I wrote the arguments you should use somewhere in the GAN discussion. Did you use them?

08:51 < kris1> Yes the same parameters..My time to convergen is around 16min.

08:52 < kris1> *convergence

08:53 < lozhnikov> The time depends on the system. If you didn't modify anything you should get the same results

08:54 < kris1> I will send you the results i am getting along with the snapshot of the loss functions

08:57 < kris1> I have create a PR for ResizeLayer now i will debug the error once the execution for gan.cpp finishes.

09:19 < lozhnikov> kris1: I uploaded the code here https://github.com/lozhnikov/mlpack/tree/gan2

09:20 < lozhnikov> compile the following file: https://www.irccloud.com/pastebin/TDMmdZ0S/gan.cpp

09:20 < lozhnikov> and run the test: LD_LIBRARY_PATH=lib ./gan_test -i digits_train.arm -o ../../digits-results.txt -v -s -e 500 -r 0.1 -g 2 -N 32 -b 10 -G 128 -D 32 -t 0

09:22 < lozhnikov> then do the following in python:

09:22 < lozhnikov> import numpy as np

09:22 < lozhnikov> from matplotlib import pyplot as plt

09:22 < lozhnikov> import sys

09:22 < lozhnikov> a = np.loadtxt(sys.argv[1])

09:22 < lozhnikov> plt.imshow(a, cmap='gray')

09:22 < lozhnikov> plt.show()

09:22 < lozhnikov> apply this script to ../../digits-results.txt

09:23 < lozhnikov> you should get the following picture: https://usercontent.irccloud-cdn.com/file/CHKkw4mf/digits-results.png

09:24 < lozhnikov> kris1: I'll look through the ResizeLayer PR today

09:25 < kris1> Okay sure…. i am not pretty sure about the error in ReseizeLayer…..symbol not found for Linear.hpp layer ?? seems strange…..

09:30 < kris1> Ahh i think it might be working now(gan)…. my learning rate was wrong….stupid mistake….

09:58 < kris1> I was able to the results now for the gan.cpp :)

09:59 < kris1> Just curious to know did you try it out for other parameters as well. Basically having a higher number of iterations….

10:03 < lozhnikov> I spent 2 days in order to obtain these arguments and get these results. That's the best result that I obtained.

10:03 < lozhnikov> I think it is possible to improve that

13:47 < kris1> lozhnikov: I started testing gan implementation with the train4 dataset using the architechture from keras adverserial example.

13:47 < kris1> It’s peculiar how we are getting all gradients as zeros all the time.

13:52 < kris1> I just wanted to get gan working before testing it out on the orilley example.

13:56 < lozhnikov> I didn't look at the keras example

13:58 < lozhnikov> but many examples use the Adam optimizer (I replaced that by mini-batch SGD since the mlpack version of Adam hasn't got mini-batch support)

13:58 < lozhnikov> so, I guess our architecture differs from that example

13:59 < lozhnikov> in that case it is reasonable to try to vary some arguments

14:00 < lozhnikov> maybe it is reasonable to minimize -log(D(G(z))) instead of log(1-D(G(z)))

14:00 < lozhnikov> I mean to say that there is no quick answer

14:04 < kris1> Hmmm….. What i trying to do is to try to see if i can get the discriminator preTrained first without Gan that would give me a idea about the kinda parameters that would be required.

14:06 < kris1> Here is the example btw https://github.com/bstriner/keras-adversarial/blob/master/examples/example_gan.py

14:50 < kris1> In minibatch desecent if we have no learning at all then overallObjective == lastObjective. That would actually cause the program to terminate saying that we have converged which would be incorrect.

15:36 < lozhnikov> kris1: If I remember right there was a trivial workaround: you could use negative tolerance

15:40 < kris1> Even pretrain of discriminator is giving me 0 gradients.

15:41 < kris1> Here is the gist of what i am doing now....https://gist.github.com/kris-singh/74b8e25fcdce5dbe237b86e07a96f5ea

16:08 < lozhnikov> kris1: I think you did an error in the test.I inserted your network structure into my test and got the following: https://www.irccloud.com/pastebin/mDSXvv67/gan.cpp

16:08 < lozhnikov> I tried to run the test with the following arguments:

16:08 < lozhnikov> LD_LIBRARY_PATH=lib ./gan_test -i ~/Programming/mlpack/mnist.arm -o ../../output-mnist10000.txt -m 10000 -e 5 -r 0.1 -b 10 -g 2 -N 100 -G 2048 -D 320 -s -v

16:09 < lozhnikov> The gradients aren't equal to zero

19:41 < kartik_> <zoq> CMAES is fixed too. finally ! :)

19:43 < kartik_> waiting for your final code review.