00:31 < Suryo> zoq: A quick question regarding PSO in ensmallen. As per a previous discussion, the consensus was that in the most plain implementation of PSO, we should probably not include the gradient descent hybrid.

00:32 < Suryo> Since we don't have to compute the gradients within vanilla LBestPSO and just evaluate every point in the swarm, would it be okay to cast an input objective function as ArbitraryFunctionType instead of FullFunctionType?

00:33 < Suryo> Chintan's code casts the input function to FullFunctionType. I don't think that we actually need it to begin with...

00:34 < Suryo> And another general question and this is probably really stupid but I don't fully know know how the process of licensing and authorship of code works.

00:35 < Suryo> More specifically: right now, I've read Chintan's code, tweaked it and moved it to a branch of my fork of ensmallen. So if I submit a pull request, would the author of the code be only Chintan, or only me, or both?

00:36 < Suryo> Once again, I'm sorry if this last question is really stupid but I need some insights. Thank you.

08:45 < jenkins-mlpack2> Project docker mlpack nightly build build #231: STILL UNSTABLE in 3 hr 31 min: http://ci.mlpack.org/job/docker%20mlpack%20nightly%20build/231/

11:47 < zoq> Suryo: Using ArbitraryFunctionType instead of FullFunctionType makes sense.

11:47 < zoq> Suryo: I thinks it's fair to list Chintan and you as the authors, I see this as some sort of collaboration. Once the PR is open, we should ask him for input. Let me know what you think.

11:54 < Suryo> zoq: Once again, thank you for your response on this. For the most part, what I've been really learning through this process is about the codebase.

11:55 < Suryo> Honestly, as a plain implementation, everything that was a part of the mlpack codebase was enough -- I didn't have to do much in a technical sense.

11:56 < Suryo> But of course, this is just a starting point and I'm already considering how to bring in constraints with PSO. I'll be gathering my ideas soon.

11:56 < zoq> Suryo: It's a really nice project to get familiar with the codebase.

11:56 < Suryo> Now that you've told me about the code authorship, I will be pushing my changes to GitHub.

11:56 < zoq> Suryo: Looking forward to review/merge PSO into the repo.

11:56 < zoq> Great :)

11:57 < Suryo> But before I submit the PR, I still want to see how to get more flexibility into this...

11:57 < Suryo> For instance, PSO is something that should work for any arbitrary objective.

11:58 < Suryo> But what happens if we want to do a PSO and gradient descent hybrid? In that case, we have to make sure that the function is a differentiable function.

11:58 < Suryo> Also, do we want to have the GD hybridization as a part of PSO, or do we want to split it into two parts at a high level?

11:59 < Suryo> zoq, I think that you missed a few of my chat logs from a few days back. There were a lot of important discussions going on and I was trying to reach you though it. It was on 6th feb

12:00 < Suryo> So yes a jist... At point, arbitrary function type is working, I got Chintan's PSO to work within ensmallen. And I am guessing that the only thing remaining in between now and submitting the PR is writing the tests.

12:00 < Suryo> But that's not going to be a problem.

12:01 < zoq> Suryo: I think it makese sense to split the two, gradient and non gradient.

12:01 < zoq> Ohh, I think I did miss the messages, let's see.

12:01 < Suryo> I agree. I'll take some time to understand how to incorporate different function types for different variants of PSO. So just give me a few more days. I will be submitting a PR soon.

12:02 < Suryo> Also, there were a couple of mistakes I made previously with regards to understanding how the optimum point is being returned. So don't worry about that when you read it up.

12:06 < Suryo> Since this is still quite early, and it hasn't been too long that I've been working on it, I actually don't want to rush it. Different variants of PSO might work on different function types.

12:06 < Suryo> So I want my inputs to be such that incorporating different update and initialization methods becomes easy.

12:07 < Suryo> Thank you again for your response, zoq. I'll keep you updated.

12:09 < zoq> About the particle class, agreed ideally we can use all the nice functionality that armadillo has, so if we can avoid an extra class we should probably do that.

12:09 < zoq> On some level you can configure armadillo to use OpenBLAS, which does some parallelization; another idea might be to use OpenMP, we did this in some places, see https://github.com/mlpack/mlpack/blob/2326bf8bd0b380c17eed6b5b1c34fbfab1563774/src/mlpack/methods/reinforcement_learning/async_learning_impl.hpp#L100 for an example.

12:09 < zoq> Really like all the thoughts you put into the idea.

12:10 < zoq> Agreed, no need to rush anything.

12:15 < Suryo> Okay, so here are the things I'll be doing next. Mostly, just gathering my ideas on (i) abstraction of objectives and optimization procedures (ii) parallelization.

12:15 < Suryo> Thanks again! :)

12:15 < zoq> Sounds good :)