00:13 < Suryo> zoq: some updates. Here's a consolidated document on termination strategies for PSO. https://web2.qatar.cmu.edu/~gdicaro/15382/additional/CompIntelligence-Engelbrecht-ch16.pdf

00:14 < Suryo> On page No. 298 (10/70 in PDF), you will see a list of termination strategies. I was wondering if it would be wise to incorporate these within the existing framework. For instance, we already have termination strategies based on 'maxIter' and 'no improvement'

00:15 < Suryo> Two additional strategies would be to observe improvement over a certain number of iterations and make a decision, or to see if the normalized radius of all the particles is below a tolerance level.

00:16 < Suryo> We can do any of the following: (i) incorporate one of these into PSO (ii) incorporate both of these into PSO [because, why not?] (iii) templatize the termination strategies

00:17 < Suryo> Let me know how you would like to proceed. If you like (iii) then we should spend a little time discussing the policy design.

00:17 < Suryo> Thanks!

07:23 < Suryo> zoq: I actually take that back. After some contemplation, I think that it would be best to stick to our original plan of having PSO's performance evaluated over the most recent k epochs.

07:24 < Suryo> The reason is that in order to use the radius-normalization method, we would have to identify the point that is furthest away from the swarm leader. That would require iterating over all the particles and finding the point that is furthest away.

07:24 < Suryo> And although we can default this to an L2 norm, there is actually no particular reason to.

07:24 < Suryo> Additionally, there's the curse of dimensionality..

07:25 < Suryo> So I think that it would be best to stick to the original plan. Let me know what you think and based on that I'll implement whatever is needed

08:07 < jenkins-mlpack2> Project docker mlpack nightly build build #385: STILL UNSTABLE in 3 hr 53 min: http://ci.mlpack.org/job/docker%20mlpack%20nightly%20build/385/

11:28 < favre49> zoq Good news, after implementing RK4 in the double pole cart test, NEAT is able to solve it, and lasts 10,000 timesteps (I only allowed it that long)

11:30 < favre49> I think in the unit test, we should let the initial state be at 1 degrees, instead of the 4.5 degrees in the paper, since it takes far longer to cross 500 time steps, and may not perform too well every time.

11:30 < favre49> This is also documented in sharpneat, which is why they also use one degree as the start in their testing, so it's not a bad sign I don't think

11:34 < favre49> I'll open a PR for the fix soon. Should I also include the learning tests for the environments, or should I focus on finishing up the NEAT PR first?

16:01 < abernauer> rcurtin: So I' am making some progress with a promising error message. unable to load shared object '/home/andrew/Documents/tests/source/pca_binding.so': /home/andrew/Documents/tests/source/pca_binding.so: undefined symbol: _ZNK5boost15program_options22error_with_option_name4whatEv

17:22 < jeffin143> rcurtin , zoq : How does the blog get updated as soon as someone pushes, I mean did u use any hook ? how is the whole preocedure done

17:22 < jeffin143> procedure *

17:30 < akhandait> sreenik[m]: Hey!

17:32 < sreenik[m]> akhandait: Hey

17:33 < akhandait> Can we start?

17:33 < sreenik[m]> Yeah sure

17:33 < akhandait> Okay, so good news that they replied.

17:34 < sreenik[m]> Yes that's certainly some good news

17:34 < akhandait> Okay, that's for the mlpack-onnx translator. What's the status on the onnx-mlpack translator?

17:34 < akhandait> I think we should finish that first

17:35 < sreenik[m]> Yup I think we can go on a continuous improvement process on that since the basic structure is fixed (I have updated the file in github)

17:36 < sreenik[m]> That's because we cannot possibly test it on all possible models at once.

17:36 < akhandait> Did you solve that issue with Add operators?

17:37 < sreenik[m]> There are some onnx models in the onnx model zoo. I am going through them and trying to make them compatible

17:37 < sreenik[m]> I think I have done something with it. What I have done is that I have merged some known combinations into a single layer

17:38 < akhandait> Hmm, I think we should definitely start with mlpack-onnx within 10 days.

17:38 < sreenik[m]> Currently there is just one known combination, which is matmul + Add. So I have created a map where we can store these and the program will take care of the rest

17:38 < akhandait> I would suggest for now, don't try with the onnx model zoo models. Just simple support for Linear and Conv is fine

17:39 < sreenik[m]> Yes that is done alright. Simple linear and conv are getting converted

17:40 < akhandait> Hmm, I can think of only two important combinations: matmul and matmul + Add. Can you map matmul to LinearNoBias layer?(when there's no Add after it)

17:40 < sreenik[m]> Yes I have done that too

17:40 < akhandait> Great

17:41 < sreenik[m]> Actually if you go through the code you can see a number of data structures being used for different purposes. As we find new combinations and stuff we caan just add a few elements to it and it will be fine

17:42 < sreenik[m]> Especially the function called generateModel()

17:42 < akhandait> Okay so one thing, I think for now, you should focus on testing and structuring the onnx-mlpack code properly and get it ready for merge. No need to add support for more layers right now. Just make sure that simple mnist conv model you created works. The models from the onnx zoo would probably be complex and we should start with mlpack-onnx first.

17:43 < sreenik[m]> Yes I agree with it

17:43 < akhandait> Okay, I will go through the changes tomorrow.

17:43 < sreenik[m]> Sure

17:44 < akhandait> Hmm, and did you figure out the reshaping in onnx?

17:46 < sreenik[m]> Not quite. That is actually one reason why I am afraid to go for the mlpack-onnx translator. I think if we go for something like mlpack-onnx via libtorch we can get a solid API from torch and while converting from torch to onnx, onnx will figure out what to do

17:46 < akhandait> Hmm, I see.

17:47 < sreenik[m]> Need to go through that in detail before proceeding though. I think I will do that tonight so that we can take a decision as fast as possible

17:48 < akhandait> In this case though(onnx-mlpack), we need to support reshaping for models with both conv and linear layers.

17:48 < akhandait> I will try and see tomorrow what we can do.

17:49 < sreenik[m]> Yeah sure. Just one doubt..

17:49 < akhandait> Sure, go on

17:50 < sreenik[m]> Can there generally be any occasion where the user explicitly decides to use a reshape layer?

17:50 < akhandait> Let me think

17:51 < sreenik[m]> If not, then we can treat it as an internal manipulation and ignore it (or rather find out what layers cause the reshape and add it to the mergeable layers)

17:51 < sreenik[m]> Yea sure. Take your time and let me know

17:54 < akhandait> Personally, I have only used Reshape for going from Linear-Conv and vice versa. Mlpack does it on it's own, so I think we could treat it as an internal thing and ignore it.

17:55 < sreenik[m]> Even I think so

17:56 < akhandait> So, go forward with it then. We will see if some issue comes up

17:57 < akhandait> About the conv layer, is it working now, except the groups parameter of course?

17:57 < sreenik[m]> Yes. I'll proceed it that and keep an eye on the mlpack-onnx translator meanwhile

17:58 < akhandait> Yeah, I guess we have until we finish this one to think what to do with mlpack-onnx. Even I will explore our options.

17:58 < sreenik[m]> Yes, sounds good

17:59 < akhandait> About the conv layer, is it working now, except the groups parameter of course? I think you missed this message. :)

17:59 < sreenik[m]> Oh yes it is

18:00 < sreenik[m]> I have a converted model too in the temp section of the translator repo

18:00 < akhandait> Awesome!

18:00 < sreenik[m]> :)

18:01 < akhandait> That means if we treat the reshaping as internal in mlpack, we can convert the mnist_conv model you created. Am I right?

18:01 < sreenik[m]> Yes right

18:02 < sreenik[m]> But I am yet to run the mlpack model and see if the accuracy matches

18:02 < akhandait> So, till when do you think you can do this?

18:02 < akhandait> Oh, okay, no problem

18:02 < sreenik[m]> Wouldn't take much time I suppose. Testing the accuracy will hardl take an hour if everything is fine]

18:04 < akhandait> Just let me know when it's matching. I think we are close to merging the onnx-mlpack translator now.

18:04 < sreenik[m]> In case there is a problem with the accuracy it will take an additional couple of hours (hopefully) to debug

18:04 < sreenik[m]> Yup!

18:04 < akhandait> Will just need some reviewing and testing

18:04 < sreenik[m]> Yes

18:05 < akhandait> You would need to come up with a unit testing file like the ones in mlpack/tests.

18:05 < akhandait> That will take some time I think

18:05 < akhandait> There would be a lot of tests

18:06 < travis-ci> robertohueso/mlpack#34 (mc_kde_error_bounds - a677fa3 : Roberto Hueso Gomez): The build is still failing.

18:06 < travis-ci> Change view : https://github.com/robertohueso/mlpack/compare/e4d507d45072...a677fa3ce88a

18:06 < travis-ci> Build details : https://travis-ci.org/robertohueso/mlpack/builds/558293267

18:06 < sreenik[m]> Oh yes. I've never done it so yes it will take some time

18:07 < akhandait> Yeah, you will learn a very important programming skill next week then. :) Good luck!

18:08 < sreenik[m]> Haha

18:09 < akhandait> Can you push the latest changes to the repo by tomorrow(including the reshaping) so that I can play around with a little.

18:09 < sreenik[m]> I think it's already there. But I'll check it once more and push it again by tonight

18:10 < akhandait> Last thing, for now you are just using the default value for groups(=1), right?

18:12 < sreenik[m]> Technically yes, I mean, since mlpack doesn't support the parameter yet, I didn't need to set anything.

18:13 < akhandait> Yup, and what about dilations? Are you using the AtrousConvolution layer when they are not 1?

18:15 < sreenik[m]> Ohh thanks for reminding. I had forgotten about that. A small change probably. Anyway, can I just use atrous conv instead of conv keeping dilations as 1 when converting a normal conv layer?

18:17 < akhandait> Hmm, I see no reason why that won't work. Go ahead with it.

18:18 < sreenik[m]> Okay that should make it a lot easier

18:19 < akhandait> Okay then, I gotta go now. I will get back in a couple of days after looking at the changes.

18:19 < akhandait> Good night:)

18:20 < sreenik[m]> Sure, good night :)

18:28 < abernauer> rcurtin: Update the code was seg faulting and I ran it through gdb. The issue is armadillo related. Program received signal SIGSEGV, Segmentation fault.arma::arma_warn<char [28]> (x=...) at /usr/include/armadillo_bits/debug.hpp:316316arma_warn(const T1& x)

20:57 < zoq> Suryo: I agree, let's stick to the plan, implementing other termination strategies is still a valid point we should look into once the PSO is in a good state. Also, there is https://github.com/mlpack/ensmallen/pull/119 which provides the possibility to add callbacks which also covers termination strategies: https://github.com/mlpack/ensmallen/pull/119/files#diff-040f465a8e40b0232db6f1d379e56642 is just a

20:57 < zoq> simple example.

20:57 < zoq> favre49: That is great news indeed; Using an already optimized initial state for the test sounds reasonable to me, not too easy since we still like to see if the method works.

20:57 < zoq> favre49: It might be a good idea to work on the learning tests for the environments as well, yes.

20:57 < zoq> favre49: Btw, really enjoyed reading the latest update.

21:03 < zoq> jeffin143: Yes, we use a webhook; each commit triggers a Jenkins build: http://ci.mlpack.org/job/blog/ which builds the static html pages using a bunch of bash and python scripts: https://github.com/mlpack/blog/tree/master/script/doxygen-src. The basis is still the Doxygen build, similar to what we use for the documentation, or I guess what we used for the documentation.