08:11 < Aakash-kaushikAa> Sure, No problem with that.

12:05 < jonpsy[m]> @zoq Hi, do you have a minute? So I created a [test file](https://gist.github.com/jonpsy/46b8ad50b8120172830b734e87dddde6) for generating csv and its working great on local machine. I'm not sure how to compile cpp on binder instance, particularly the "python" linking part.

12:05 < jonpsy[m]> After installig pandas-datareader and setting up properly. The ```pFunc = Py_GetAttrString(...``` is returning NULL :(

12:15 < jonpsy[m]> .. which is returning "Call Failed" error

12:19 < jonpsy[m]> * @zoq Hi, do you have a minute? So I created a [test file](https://gist.github.com/jonpsy/46b8ad50b8120172830b734e87dddde6) for generating csv and its working great on local machine. I'm not sure how to compile cpp on binder instance, particularly the "python" linking part.

12:19 < jonpsy[m]> After installig pandas-datareader and setting up properly. The ``` PyCallable_Check(pFunc)``` returning NULL :(

14:14 < zoq> jonpsy[m]: Will look into it.

14:16 < zoq> jonpsy[m]: About the compiling question, you don't it's handled by xeus cling.

14:16 < jonpsy[m]> Actually xeus-cling notebook isn't verbose enough on errors

14:17 < jonpsy[m]> when I was facing errors on my local machine, the python errors should up concisely.

14:18 < jonpsy[m]> I reckoned if I could run my test.cpp, I can clearly find out the python errors

14:20 < zoq> Above you mentioned it works fine on your local machine, but not on the binder instance?

14:20 < jonpsy[m]> yep

14:21 < jonpsy[m]> point to remember, I used this (script)[https://gist.github.com/jonpsy/46b8ad50b8120172830b734e87dddde6]

14:21 < jonpsy[m]> * point to remember, I used this [script](https://gist.github.com/jonpsy/46b8ad50b8120172830b734e87dddde6)

14:22 < zoq> Have you tested the Python code in a Python notebook as well?

14:23 < zoq> Currently building your branch with binder, so I can test it in some minutes as well.

14:23 < jonpsy[m]> do you mean running locally using xeus cling

14:25 < zoq> I mean, using lab.mlpack.org point it your repo and branch, open-up a python notebook and see if https://github.com/mlpack/examples/blob/231c19bc28f6de535c95a78cfb1546eb27495f79/utils/portfolio.py runs fine.

14:26 < jonpsy[m]> Thanks, I'll be on it.

14:36 < jonpsy[m]> zoq: it works perfectly fine on Pyton notebook.

14:37 < jonpsy[m]> * zoq: it works perfectly fine on Python notebook.

14:37 < jonpsy[m]> brb dinner

14:38 < jonpsy[m]> * zoq: it works perfectly fine on Python notebook (I used custom inputs)

14:50 < zoq> jonpsy[m]: This is what I used inside a C++ notebook - https://gist.github.com/zoq/988cae9ed05844d95826d928d453cfc0

14:50 < zoq> jonpsy[m]: Which runs just fine and generates portfolio.csv

14:52 < zoq> Also, make sure to run it inside a folder and not in the root folder, because PyRun_SimpleString("sys.path.append(\"../utils/\")"); fail fail otherwise.

15:07 < Aakash-kaushikAa> Btw have we thought about improving the documentation through this https://developers.google.com/season-of-docs/docs

15:08 < Aakash-kaushikAa> If we are selected as an orginastion we can get good documentation and the other person does gets all the benifit provided by the program

15:10 < zoq> Aakash-kaushikAa: We thought about it, but hadn't had time in the past.

15:10 < Aakash-kaushikAa> Why i am saying this is because when i have to look something up i prefer looking it in the codebase rather than documnetation just becasue of the fact that this is far more easier. and the documentation looks a bit odd even by todays standards.

15:11 < jonpsy[m]> > jonpsy: This is what I used inside a C++ notebook - https://gist.github.com/zoq/988cae9ed05844d95826d928d453cfc0

15:11 < jonpsy[m]> Isn't this the test.cpp which I sent previously?

15:11 < Aakash-kaushikAa> But that is like the first thing we need to do so it's more inviting to users

15:11 < Aakash-kaushikAa> when i am looking at a new ml or dl library i go their tutorials and then refer documentation very heavily.

15:11 < Aakash-kaushikAa> go to*

15:12 < Aakash-kaushikAa> I mostly don't have a need to visit PyTorch's or Tensorflow's github

15:13 < Aakash-kaushikAa> i get the part where you are bored about people just talking about it and maybe it just get's left out.

15:15 < Aakash-kaushikAa> https://readthedocs.org/ I like these too. and i think it is actually free for open source projects without and hidden charges or anything

15:16 < zoq> jonpsy[m]: Close, except that I included #include <mlpack/xeus-cling.hpp> and run the code from within a subfolder.

15:17 < jonpsy[m]> I did that too

15:17 < jonpsy[m]> that's how i came up with the "../utils" idea.

15:17 < zoq> Aakash-kaushikAa: You are right, and we touched on the topic - https://github.com/mlpack/mlpack/issues/2922 as well.

15:18 < jonpsy[m]> I reckon you compiled it in binder instance?

15:18 < zoq> jonpsy[m]: Yes, I just opened a C++14 notebook, and used the code I linked to.

15:18 < zoq> jonpsy[m]: It dosn't work for you?

15:19 < jonpsy[m]> Do you mind testing this.., wait I'll send you

15:19 < jonpsy[m]> Let me know if you get "Call failed" erorr

15:23 < zoq> jonpsy[m]: The order of the arguments you pass to the Portfolio function isn't correct.

15:23 < jonpsy[m]> WHAAAAAAAAAAAAAAAAAAAAAAAAAAA

15:24 < jonpsy[m]> You're correct.. So I wasted 3 hours on arg ordering :(

15:25 < zoq> Wondering if the python code returns any errors.

15:26 < jonpsy[m]> no it doesn't.

15:26 < jonpsy[m]> Lemme try specifying custom filepath, that'll be the nail in the coffin. Then it's GTM I suppose?

15:26 < rcurtin[m]> Aakash-kaushik (Aakash kaushik): yeah, we've thought about it before, the problem is, it would take a lot of time for someone to oversee it, and I don't think anyone has had the time

15:27 < rcurtin[m]> but I totally agree, it would be great if users could mostly follow examples and tutorials on the website, and never need to dive into the source

15:27 < rcurtin[m]> unfortunately we don't have the kind of resources that Google or Facebook pour into TF or PyTorch 😃

15:28 < rcurtin[m]> but, I do think this is something that will definitely be improved soon (in part by your project and others this year!)

15:29 < jonpsy[m]> > Lemme try specifying custom filepath, that'll be the nail in the coffin. Then it's GTM I suppose?

15:29 < jonpsy[m]> Done! Custom path works as well. Yipee

15:30 < zoq> jonpsy[m]: Nice!

15:56 < zoq> Aakash-kaushikAa: Btw. I made some progress with the windows issue, in the sense that I can reproduce it locally.

15:58 < shrit[m]> Aakash-kaushik (Aakash kaushik): Totally agree with rcurtin on this point, we need to provide more and more documentations, and please when ever you noticed something is missing, either open a pull request or an issue so we can keep track of it, since we might forget where the missing things are

16:00 < shrit[m]> We need to have some kind of an open documentation issue, where we update constantly the keep tracking of missing parts that need to be added, this will allow new contributors to pick from the list and contribute to mlpack., basically the same way we have done with activation layers,

16:01 < shrit[m]> * We need to have some kind of an open documentation issue, where we update it constantly and keep tracking of missing parts that need to be added, this will allow new contributors to pick from the list and contribute to mlpack., basically the same way we have done with activation layers,

16:21 < jonpsy[m]> +1, I'd say the issue should be opened on mlpack/examples.

16:23 < shrit[m]> I am not sure if there any active documentation in `mlpack/examples`, I think all of mlpack docs are on the mlpack main repository

16:25 < jonpsy[m]> My line of reasoning is that, since mlpack/examples are meant to be as an "introduction" to the library with ready to use examples. Its only fitting to have extensive documentations of each method there.

16:25 < jonpsy[m]> Ofcourse, do correct me if my vision's limited

16:27 < rcurtin[m]> jonpsy: one of the things that makes it tricky is that we have interfaces in lots of different languages

16:27 < rcurtin[m]> I think for the bindings for other languages, the generated documentation is pretty good: https://www.mlpack.org/doc/mlpack-3.4.2/cli_documentation.html

16:27 < rcurtin[m]> but what would be even more awesome is if we could link each method with an example from the examples repo or something :)(

16:28 < jonpsy[m]> that'd be cool!

16:28 < rcurtin[m]> it's likely that we could redo our C++ documentation in a more clear way, but I'm not exactly sure what the best way to do it would be. it would be a big undertaking

16:28 < rcurtin[m]> right now doxygen (or sphinx if we switched to that) does an ok job of showing the API and functions of each class, but it can be hard to navigate and doesn't make it clear what the "important" classes are and what the "support" classes are

16:29 < rcurtin[m]> shrit: nice PRs that you just opened, I'll try to review them shortly 😃

16:29 < shrit[m]> You are welcome, I did not check for the installation one on my machine, let us see if it works correctly on the CI

16:30 < rcurtin[m]> yeah, for #2952 I will probably run it locally in a container with only a C++ compiler and openblas installed, then install it and make sure everything is there :)

16:33 < zoq> Agreed, also I encountered another issue where the first cmake pass failed (because it couldn't find some dependencies) but the second pass was sucessfull, since the auto-downloader downloaded some files.

18:36 < Aakash-kaushikAa> Hey @zoq great on the windows issue that you could reproduce, was it just because of the permissions or something else too ?

18:36 < rcurtin[m]> zoq: I've been playing with the `ann-vtable` branch... on this branch, each layer stores its own weights separately, but I think ensmallen would require that all weights are stored in a single `arma::mat` (or equivalent)... so I think maybe we will have to ensure that we set each layer's weights to be an alias before merge?

18:37 < zoq> Aakash-kaushikAa: Not sure yet, what the issue is, I'll continue my debugging session later today :)

18:39 < Aakash-kaushikAa> And while coming back to the documentation part I agree to all of this and as @zoq said this could be an issue and can be worked slowly like activation layers I think we should decide on a clear path and break that down into smaller ones so new comers can contribute to that and it is easy for us to maintain and actually reach somewhere because i know it's going to be huge but if we don't break it down and start and

18:39 < Aakash-kaushikAa> let others know that we have this problem maybe as a good first issue it won't really go anywhere.

18:39 < Aakash-kaushikAa> also sorry for the big paragraph 😛

18:41 < Aakash-kaushikAa> > @Aakash-kaushik: Not sure yet, what the issue is, I'll continue my debugging session later today :)

18:41 < Aakash-kaushikAa> sure, i tried changing up the install prefix and seeing if i could find something for cmake_install_prefix in mlpack's cmake file because that doesn't errors out while installing but no success there too.

18:41 < zoq> rcurtin[m]: Yes, I don't have a good solution for that, the issue I have with the current solution is that all weights have to be kept in memory.

18:41 < rcurtin[m]> yeah, you mean that this prevents out-of-core learning?

18:42 < zoq> yes, am I wrong?

18:45 < rcurtin[m]> no, I think you're correct, you'd have to write some interesting shims to make it work otherwise

18:45 < rcurtin[m]> let me think about this a minute, because really it would be great to support that

18:45 < zoq> Aakash-kaushikAa: Sounds like a good idea to me, we can use the issue I mentioned above to figure out what our options are and then we can go from there.

18:46 < zoq> rcurtin[m]: I guess that would mainly be interesting for ressource constrained devices.

18:47 < rcurtin[m]> yeah, there is some trickiness. ensmallen expects some kind of parameters matrix that matches the Armadillo API, and ensmallen will expect to be able to compute the objective and the gradient based on those parameters (and then it will update the parameters)

18:49 < rcurtin[m]> inside of `Evaluate()` and `Gradient()`, we can be careful with how we access the parameters (in fact that will generally be one layer at a time), but when the optimizer does some linear algebra expression to update the weights, we will not have control over how memory is accessed (if we are just using an `arma::mat`, that is)

18:50 < rcurtin[m]> it almost seems to me like the "right" way to do this would be to have some kind of `CollectionOfMatrices` "shim" class (I chose an awful name, we can do better :)), which presented the same API as Armadillo, but under the hood perhaps was a little bit smarter about memory handling, loading/unloading individual matrices as necessary to handle memory pressure

18:50 < zoq> Right, I guess that is not something we should fix on the mlpack side, wondering if Conrad thought about it.

18:50 < rcurtin[m]> another thing that could be done is just the `mmap()` trick, which actually should be pretty easy for FFN parameters

18:51 < rcurtin[m]> (in this situation, you create a big mmap-ed file, then open it with `mmap()` and pass the given memory pointer to the Armadillo advanced constructor. then, you just hope the kernel brings things in and out of RAM in a reasonable way :))

18:52 < zoq> That is POSIX only right?

18:52 < rcurtin[m]> `mmap()` could work well if all operations on it are in-place, which at least for SGD-type optimizers should be true

18:52 < rcurtin[m]> yeah, so it wouldn't work on Windows

18:52 < rcurtin[m]> but I think Windows has similar functionality

18:52 < zoq> Hm, that sounds like a reasonable solution to me.

18:53 < rcurtin[m]> birm and I played with this some years ago, and then another student took over the effort; in part this culminated in a nice workshop paper: https://dl.acm.org/doi/pdf/10.1145/2882903.2914830

18:54 < rcurtin[m]> in any case, the mmap solution would require all the parameters to "seem" contiguous inside of one Armadillo matrix, so I guess we would still need some way to force all the layers in the `ann-vtable` branch to use the same memory

18:55 < rcurtin[m]> I'm trying to get the serialization tests to pass, so in order to do that I'll first do a big hack (copy out each layer's parameters during `EvaluateWithGradient()`) just to make it work, and then we can make that more efficient later

18:55 < zoq> Sounds good to me.

18:57 < zoq> I guess if it works for one, it's easy to do the same thing for the other layers as well.

18:58 < zoq> The M3 paper mentioned that it was using mlpack for testing.

18:59 < zoq> With a minimal modification :)

19:00 < rcurtin[m]> yeah, the 'minimal modification' was just creating an `arma::mat` with the advanced constructor using `mmap()` :)

19:00 < rcurtin[m]> I think maybe L-BFGS needed to be modified a little bit, but I can't remember

19:00 < rcurtin[m]> that `mmap()` hack doesn't work if, e.g., you try to copy the matrix (it'll be too big) or do any operations that cause really big intermediate matrices or something

19:01 < zoq> Honestly that sounds like a really simple and neat idea to me.

19:01 < rcurtin[m]> it won't work for GPUs, but we can figure out what to do with GPUs some other day :)

19:01 < rcurtin[m]> we do have a decent amount of flexibility since ensmallen just requires some type that implements the Armadillo API

19:02 < rcurtin[m]> so, a "wrapper class" that, e.g., holds weights for different layers on different GPUs could be reasonably possible

19:04 < zoq> Right, good thing the armadillo API is simple.

19:05 < rcurtin[m]> yeah, and in the worst case we can define an implicit conversion to some Armadillo type for, e.g., decompositions and similar

19:05 < zoq> Sure, in ten years :)

19:05 < rcurtin[m]> someday when time permits 😃