03:53 <EshaanAgarwal[m]> * checking the forward and backward function

07:39 <jonathanplatkiew> Thanks ryan>! I saw that though. What confuses me is that SigPack (in their documentation) stipulates that `arma::Col<T` with any type for `T` is an option. But it seems that is not, which is misleading.

08:40 <jonpsy[m]> <EshaanAgarwal[m]> "Sure ! I am checking the..." <- So we're sure that inited weights of both are same? Do you have any screenshots or data to share?

11:14 <rcurtin[m]> maybe it's worth filing a bug report with the sigpack developers?

11:57 <jonathanplatkiew> 👍

14:14 <rcurtin[m]> zoq: oops, I meant to have [#3270](https://github.com/mlpack/mlpack/pull/3270) merge into the branch for [#3269](https://github.com/mlpack/mlpack/pull/3269), so the diff was accidentally huge, but really the diff for #3270 should be [just this](https://github.com/mlpack/mlpack/pull/3270/commits/22fd012bcc7139f412f7068e7e7bd64b2a10a411) 😃

14:52 <EshaanAgarwal[m]> <jonpsy[m]> "So we're sure that inited..." <- i had one question. lets say we were able to make everything same in pytorch and mlpack network including weights. We still wont be able to make equal state that we get from the environment in both implementations ?

14:52 <EshaanAgarwal[m]> EshaanAgarwal[m]: zoq:

14:55 <zoq[m]> EshaanAgarwal[m]: True, but we can still make sure it's the same for a single run. e.g. by taking the input from pytorch, store it in a matrix and return it as part of the mlpack env.

14:56 <EshaanAgarwal[m]> zoq[m]: okay ! sure then i will incorporate that! otherwise making everything same wont do much

14:57 <zoq[m]> Yeah, we can even do it for two samples, but I would start with one since it's easier.

14:58 <EshaanAgarwal[m]> also i wanted to print results from forward and backward. I dont see any function to get that ! so should i print it in my implementation (in mlpack library) itself for the time being using `std::cout` or do we have any getter function for that ?

14:59 <zoq[m]> I would do it as part of the implementation itself (mlpack implementation), easiest solution.

14:59 <EshaanAgarwal[m]> zoq[m]: cool! thanks

15:01 <EshaanAgarwal[m]> zoq[m]: is there a way to load this in a easier way ? otherwise i would have to change the environment implementation too to incorporate loading from a vector.

15:03 <zoq[m]> I would just modify the `Sample()` function, and load the matrix and return it, and just discard any other implemetnation.

15:03 <zoq[m]> Or if you think it's easier, just not deal with the env and provide the output from pytorch.

15:03 <zoq[m]> It's somewhat the same.

15:03 <zoq[m]> But I guess the first method is easier.

15:05 <zoq[m]> https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/reinforcement_learning/environment/cart_pole.hpp#L151

15:05 <zoq[m]> is the function

15:06 <EshaanAgarwal[m]> zoq[m]: i guess you are talking `InitialSample()` here. since `Sample()`;s implementation in both implmentation is same. But initial sample that we get would differ because of random functions of both implementation

15:06 <EshaanAgarwal[m]> EshaanAgarwal[m]: and different initial sample can lead to different results.

15:07 <EshaanAgarwal[m]> https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/reinforcement_learning/environment/cart_pole.hpp#L207

15:07 <EshaanAgarwal[m]> EshaanAgarwal[m]: i meant this !

15:12 <zoq[m]> Yeah, I would "overwrite" both functions, but you can step through all the steps to make sure you get the right output.

15:14 <EshaanAgarwal[m]> zoq[m]: ok ! i will look and make necessary change.

15:22 <jonpsy[m]> Super busy with office work today; wouldn't be able to attend. If zoq you're attending the meet then I expect Eshaan Agarwal to write meeting summary. Else if no one's attending I'm still expecting some progress report.

15:24 <EshaanAgarwal[m]> jonpsy[m]: I am comfortable with both! I will share my progress for today ! i have a bit of direction for what else i have to do for now ( make sure initial sample of environment is same for both impl). We can meet maybe on tmrw and Then i would be to share you more consolidated progress.

15:25 <EshaanAgarwal[m]> > <@jonpsy:matrix.org> Super busy with office work today; wouldn't be able to attend. If zoq you're attending the meet then I expect Eshaan Agarwal to write meeting summary. Else if no one's attending I'm still expecting some progress report.

15:25 <EshaanAgarwal[m]> * I am comfortable with both! I will share my progress for today ! i have a bit of direction for what else i have to do for now ( make sure initial sample of environment is same for both impl). We can meet maybe on tmrw and Then i would be to share you more consolidated progress.

15:25 <EshaanAgarwal[m]> * I am comfortable with both! I will share my progress for today ! i have a bit of direction for what else i have to do for now ( make sure initial sample of environment is same for both impl). We can meet maybe on tmrw and Then i would be to share you more consolidated progress.

15:27 <EshaanAgarwal[m]> EshaanAgarwal[m]: because without making that same,we wont be able to make initial conditions same for both networks.

15:28 <zoq[m]> Are you able to finish it by tomorrow?

15:29 <jonpsy[m]> I mean, I can't guarantee being able to attend meets besides weekends.

15:30 <EshaanAgarwal[m]> zoq[m]: You mean changes in environment? Sure I would by then

15:30 <jonpsy[m]> Didn't we ensure initial weight are same for both? Why that topic again, im confused

15:31 <EshaanAgarwal[m]> jonpsy[m]: Weights of network. But both environment implementation use a random function to get the initial sample. Although the range from which we get the random value is same but the final random values would differ.

15:31 <EshaanAgarwal[m]> This would make the initial sample which both policy get different and therefore corresponding value and actions may or may not be different.

15:33 <EshaanAgarwal[m]> * This would make the initial sample which both policy get different and therefore corresponding value and actions will be different in virtually all of the cases.

15:33 <EshaanAgarwal[m]> * This would make the initial sample which both policy get different and therefore corresponding value and actions (that our network compute) will be different in virtually all of the cases.

15:35 <EshaanAgarwal[m]> * Weights of network. But both environment implementation use a random function to get the initial sample. Although the range from which we get the random value is same but the final random state that we sample would differ.

15:35 <zoq[m]> We should make sure, we finish the steps we discussed, which includes making sure the backward step works as expected.

15:35 <jonpsy[m]> EshaanAgarwal[m]: Can't we fix it too?

15:35 <jonpsy[m]> Seems only natural.

15:35 <EshaanAgarwal[m]> zoq[m]: Sure thing ! We would be able to that make sure only when we have the initial sample same.

15:36 <EshaanAgarwal[m]> jonpsy[m]: Sure if I save that from Pytorch and load it in mlpack implementation.

15:36 <jonpsy[m]> should be a quick one

15:37 <EshaanAgarwal[m]> EshaanAgarwal[m]: But it would be a bit messy because ! For every training run that I make, I will have to manually do it again.

15:38 <EshaanAgarwal[m]> jonpsy[m]: Will have to make changes in environment and compile it. That might take some time.

15:38 <jonpsy[m]> EshaanAgarwal[m]: We only need it for one forward & backward pass.

15:39 <jonpsy[m]> So we can set the random number fixed, such as 0.6

15:40 <EshaanAgarwal[m]> jonpsy[m]: Ok but we won't be able to control that in put pytorch implementation. So in my opinion, loading the sample from Pytorch to mlpack seems a bit more reasonable

15:40 <EshaanAgarwal[m]> * Ok but we won't be able to control that in our pytorch implementation since we use openai gym. So in my opinion, loading the sample from Pytorch to mlpack seems a bit more reasonable

15:44 <EshaanAgarwal[m]> Quite tedious. Although from what I could make out from digging in the code, I am sure that forward pass is working just fine.

15:44 <EshaanAgarwal[m]> Bug is somewhere in our `update()` as you mentioned in the previous meet. 😬

15:50 <EshaanAgarwal[m]> <EshaanAgarwal[m]> "I am comfortable with both! I..." <- zoq @marcusedel:matrix.org: would we be meeting today then ?

15:54 <zoq[m]> Let's do it tomorrow, looks like you are working on it right now, unless you have any questions.

15:55 <EshaanAgarwal[m]> zoq[m]: Cool. I don't have any. Will ask if I get them.

15:56 <EshaanAgarwal[m]> * Cool. I have a direction to work on Don't have any. Will ask if I get them.

15:57 <EshaanAgarwal[m]> * Cool. I have a direction to work on. Don't have any questions. Will ask if I get them.