ChanServ changed the topic of #mlpack to: "mlpack: a fast, flexible machine learning library :: We don't always respond instantly, but we will respond; please be patient :: Logs at http://www.mlpack.org/irc/
b1 is now known as mistiry
ImQ009 has joined #mlpack
eddelbuettel[m] has left #mlpack []
< jeffin143[m]>
@himanshu_pathak:matrix.org: woh you have added some nice improvements
< jeffin143[m]>
Deep belief , svm
< R-Aravind[m]>
Hello guys
< zoq>
R-Aravind[m]: Hello there.
< R-Aravind[m]>
I wish to help, so can you tell me what you guys are working on?
< jeffin143[m]>
There are many things we are working on
< jeffin143[m]>
You can view the GitHub issues
< jeffin143[m]>
@walragatver:matrix.org id you have the bandwidth today ?? Could you look at the last two commits
< jeffin143[m]>
May be we can finally complete today the pr curve ??
< jeffin143[m]>
There is no issue also if you don't have
< rcurtin>
zoq: thanks for the automatic video meet-up email, it does a much better job than I was doing :-D
< zoq>
rcurtin: :)
< KimSangYeon-DGU[>
kartikdutt18: sakshamb189 Are you there?
< KimSangYeon-DGU[>
Great, can you tell us the progress?
< kartikdutt18[m]>
Sure, I converted the weights of Darknet53 model, I have pushed all required changes in the PR and I'm gettting nearly the same output in Darknet53, I'm looking into why I'm not getting exactly the same output which was the case for Darknet19.
< KimSangYeon-DGU[>
Do you think it's because of the data type like uint8?
< kartikdutt18[m]>
Not really I am testing with the same tensor as PyTorch.
< KimSangYeon-DGU[>
And does the slightly different output affect the prediction?
< sakshamb189[m]>
were you able to find the fix for the preprocessing in pytorch ?
< kartikdutt18[m]>
Yes it does, on the test set that I used for Darknet 19 I got 61% in Darknet53 in PyTorch and 53% in mlpack.
< kartikdutt18[m]>
<sakshamb189[m] "were you able to find the fix fo"> In python, using np.uint8 gave nearly the same result as ToTensor().
< sakshamb189[m]>
so, there was no difference with using np.uint8? I am not sure what you mean
< KimSangYeon-DGU[>
Yes, they should be the same I guess
< kartikdutt18[m]>
Yes they were same to the degree of 1e-2.
< sakshamb189[m]>
so we don't really need to use that and we have not been able to figure out an equivalent for that in mlpack? Is that correct?
< kartikdutt18[m]>
If we iterate over the input features cast them as unit8 and divide by 255 should work. I spent most of yesterday and today reimplementing darknet 53 model in PyTorch so that access all of it's weights since it's made up of residual network followed by making corresponding changes in mlpack implementation. So I haven't checked how we would cast it but maybe that could be another PR for preprocessing that we can add
< kartikdutt18[m]>
with the converter.
< kartikdutt18[m]>
* If we iterate over the input features cast them as unit8 and divide by 255 should work. I spent most of yesterday and today reimplementing darknet 53 model in PyTorch so that the converter can access all of it's weights since it's made up of residual network followed by making corresponding changes in mlpack implementation. So I haven't checked how we would cast it but maybe that could be another PR for
< kartikdutt18[m]>
preprocessing that we can add with the converter.
< sakshamb189[m]>
the pytorch implementation you are referring, must have the preprocessing as well right?
< kartikdutt18[m]>
The preprocessing is just ToTensor() in pytorch. In mlpack it would be casting it as unint8 and dividing it by 255.
< kartikdutt18[m]>
* The preprocessing is just ToTensor() in pytorch. In mlpack it would be casting input as unint8 and dividing it by 255.
< sakshamb189[m]>
alright I see and what accuracy does the pytorch model achieve?
< kartikdutt18[m]>
<KimSangYeon-DGU[ "I thought the difference is from"> For testing I use tensor csv as input so that it doesn't affect the model performance in any way.
< KimSangYeon-DGU[>
Ok
< KimSangYeon-DGU[>
I see
< kartikdutt18[m]>
Since we first want to see if models work correctly rather so that we can merge the Darknet PR.
< kartikdutt18[m]>
* Since we first want to see if models work correctly so that we can merge the Darknet PR.
< KimSangYeon-DGU[>
Then, we can derive that some layer caused the difference
< KimSangYeon-DGU[>
on DarkNet53
< kartikdutt18[m]>
There are only two layers that are new in Darknet 53, a linear layer and residual layer / network.
< sakshamb189[m]>
I think kartikdutt18 is working on that right?
< kartikdutt18[m]>
Yes.
< KimSangYeon-DGU[>
Great
< kartikdutt18[m]>
Few differences that I already fixed are, That darknet19 used a threshold of 1e-1 whereas darknet53 uses a threshold of 1e-2 in LeakyReLU and got the parameter count to match
< sakshamb189[m]>
Do you think you can make the fix in the converter as well?
< sakshamb189[m]>
in this PR only?
< kartikdutt18[m]>
Sure, I can have a seperate PR to do that if that is what you meant.
< KimSangYeon-DGU[>
anywayt
< KimSangYeon-DGU[>
Oops
< KimSangYeon-DGU[>
Ok, then kartikdutt18 You'll work on the DarkNet53 to figure it out where the dfference is and then work on the converter, right?
< KimSangYeon-DGU[>
After that, we need to try to make preprocessed input data the same between mlpack and PyTorch.
< KimSangYeon-DGU[>
I mean ToTensor() in PyTorch
< sakshamb189[m]>
kartikdutt18: you were saying that you were able to find the preprocessing that needs to be done for ToTensor() so you should be able to implement that now?
< sakshamb189[m]>
in this PR only?
< kartikdutt18[m]>
I think the converter works for both Darknet19 and Darknet53. We already have the weights for darknet19. As needed we expand it for other layers when needed. Like I added linear layer today for Darknet53 and we work on YOLO we can add any other required layer as well. As sakshamb189 said, if we can verify if the models are correct with converter first to merge the models, would it better If we first verified and
< kartikdutt18[m]>
merged all models and then we can have a PR for converted weights and required preprocessing.
< kartikdutt18[m]>
* I think the converter works for both Darknet19 and Darknet53. We already have the weights for darknet19. As needed we expand it for other layers when needed. Like I added linear layer today for Darknet53 and when we work on YOLO we can add any other required layer as well. As sakshamb189 said, if we can verify if the models are correct with converter first to merge the models, would it better If we first verified
< kartikdutt18[m]>
and merged all models and then we can have a PR for converted weights and required preprocessing.
< kartikdutt18[m]>
<sakshamb189[m] "in this PR only?"> Sure, I would first have to try that but I think that's doable.
< sakshamb189[m]>
alright great :)
< kartikdutt18[m]>
> <@sakshamb189:matrix.org> in this PR only?
< kartikdutt18[m]>
* Sure, I would first have to try that in C++ but I think that's doable.
< KimSangYeon-DGU[>
Ok, Great
< KimSangYeon-DGU[>
Is there anything to discuss?
< KimSangYeon-DGU[>
I'm done
< kartikdutt18[m]>
<kartikdutt18[m] "I think the converter works for "> Could you let me know if this makes sense. That once I ensure that Darknet53 is correct and nothing is to be changed we could merge that. And then repeat it for the YOLO models.
< sakshamb189[m]>
yup but if we can fix the weight converter we should also try to merge that in this PR only.
< KimSangYeon-DGU[>
In my opinion, we can merge this if we verify the model is correct, but we should work on the normalization issue.
< kartikdutt18[m]>
Sure, I will first try to get the model working and I'll push all changes to the converter repo. And then I can try the normalization issue ( I think what I said above should work).
< KimSangYeon-DGU[>
Yes, the best is to load and normalize images using mlpack's feature
< KimSangYeon-DGU[>
without PyTorch
< kartikdutt18[m]>
Sure, I will add imagenet preprocessor function in preprocessing directory.
< KimSangYeon-DGU[>
Ok, thanks!
< KimSangYeon-DGU[>
Is there anything to discuss?
< kartikdutt18[m]>
I think that's it from my side. Hopefully we can merge the Darknet PR in next couple of days with the preprocessor function.
< kartikdutt18[m]>
* I think that's it from my side. Hopefully we should be able to merge the Darknet PR in next couple of days with the preprocessor function.
< KimSangYeon-DGU[>
Yes
< sakshamb189[m]>
yes alright then let's meet next week. Thanks guys!
< KimSangYeon-DGU[>
Thanks for the meeting, have a nice week!
< kartikdutt18[m]>
Great, Have a great week.
< himanshu_pathak[>
> @himanshu_pathak:matrix.org: woh you have added some nice improvements
< himanshu_pathak[>
> Deep belief , svm
< himanshu_pathak[>
Thanks jeffin143 Just trying finish them soon DBN will require a lot of work also need some improvement in kernel_svm :)