08:14 < ranjan123> Hello! zoq,rcurtin

12:24 < zoq> ranjan123: I'm able to run the code without any problems. I have to write int main() instead of just main(), but if the code compiles on your side, that's okay. Did you change the SGD optimizer somehow? Maybe you can debug the code step by step?

13:09 < ranjan123_> zoq: No I haven't changed anything in SGD optimizer!

13:10 < ranjan123_> could you please change Number of funtion to 55

13:10 < ranjan123_> and then compile ,run

13:10 < ranjan123_> *function

13:14 < ranjan123_> I have only added Log::Warn to sgd_impl.hpp to get some msg else everything unchanged

13:14 < zoq> ranjan123_: Works without any problems.

13:14 < ranjan123_> ohhk

13:14 < ranjan123_> thanks zpq

13:14 < ranjan123_> zoq

13:14 < ranjan123_> let me see

13:14 < ranjan123_> :)

13:15 < zoq> ranjan123_: If you could narrow down the error to a specific line, we could go from there.

13:17 < zoq> ranjan123_: Also does the error occur at i == 55? In the Evaluation of in the Gradient function?

13:17 < ranjan123_> no you are saying that it works without any problem, then I guess I have changed something as I was playing around with sgd.

13:17 < ranjan123_> yes

13:18 < ranjan123_> ohh i==55

13:18 < ranjan123_> not sure

13:19 < ranjan123_> let me check with fresh mlpack. then I will come back

13:20 < zoq> ranjan123_: I also checked the code, and it looks good to me, but maybe I missed something. Sounds good.

14:36 < travis-ci> mlpack/mlpack#719 (master - 460e326 : Marcus Edel): The build passed.

14:36 < travis-ci> Change view : https://github.com/mlpack/mlpack/compare/fb3994c48e59...460e326cb2dd

14:36 < travis-ci> Build details : https://travis-ci.org/mlpack/mlpack/builds/122257021

14:39 < ranjan123_> zoq: It is giving same error . I have cloned new mlpack from github then build it installed it, then ran my prog. But I have no idea Why it is running in your system .

14:41 < ranjan123_> I have compiled it this way: g++ find_equ.cpp -lmlpack -lboost_serialization -larmadillo -std=c++11

14:43 < zoq> ranjan123_: That's weird, can you go through the program step by step, to narrow down the error position? I guess, once we know the exact line, we can solve the error.

14:44 < ranjan123_> see this

14:44 < ranjan123_> Program received signal SIGSEGV, Segmentation fault. 0x00000000004095e1 in MyFunction::Evaluate(arma::Mat<double> const&, unsigned long) const () (gdb) backtrace #0 0x00000000004095e1 in MyFunction::Evaluate(arma::Mat<double> const&, unsigned long) const () #1 0x000000000040d56a in mlpack::optimization::SGD<MyFunction>::Optimize(arma::Mat<double>&) () #2 0x8000000000c53134 in ?? () #3 0x8000000000000000 in ?? () #4 0xbf1e1d

14:44 < ranjan123_> oh! wait a min

14:47 < ranjan123_> http://pastebin.com/EeyMwJHr

14:51 < ranjan123_> Compiled it with debug mode . Here you can see the error http://pastebin.com/8CSW6JsZ

14:51 < ranjan123_> at /usr/include/armadillo_bits/Mat_meat.hpp:2588 2588 arma_debug_check( col_num >= n_cols, "Mat::col(): index out of bounds");

14:52 < ranjan123_> I can't access n_cols

14:52 < ranjan123_> So that I can see the value by Printing it.

14:57 < zoq> So "index out of bounds", just to be sure 'cout<<" "<<data1.n_cols<<" "<<data1.n_rows<<endl;' return '58 3' on your side right?

14:58 < ranjan123_> yes

15:02 < zoq> ranjan123_: Okay, let's make sure you don't link against some old SGD file. Can you comment everything in Evaluate and just write 'return 1' and also comment out everything in Gradient and just write 'gradient.zeros(4);'?

15:02 < ranjan123_> ok

15:04 < ranjan123_> zoq: done ! without any error

15:08 < zoq> ranjan123_: okay, great, so I think if you just uncomment the Evaluate function it results in an segmentation fault?

15:10 < ranjan123_> hmm, yes !

15:11 < ranjan123_> uncommenting Evaluate function where ?

15:13 < ranjan123_> I have just returned 1 in the defination of Evaluate function.

15:14 < zoq> and it results in an segmentation error?

15:15 < ranjan123_> na , as I already said !

15:15 < ranjan123_> [20:31] <ranjan123_> zoq: done ! without any erro

15:15 < ranjan123_> there is no segmentation fault

15:15 < rcurtin> is there a pastebin with the code? I am happy to take a glance and see if I see anything wrong

15:16 < rcurtin> or, with the current code, that is

15:16 < rcurtin> I realize I am jumping in a little after the session has started here :)

15:16 < ranjan123_> https://github.com/ranjan1990/mlpack_optimization

15:16 < rcurtin> okay, thanks

15:16 < zoq> To be honest, everything looks good to me. So I'm kind of lost ...

15:17 < rcurtin> GetInitialPoint() is returning a four-dimensional point, but your data is three-dimensional in traj.txt

15:18 < ranjan123_> I don't know whats wrong with my system , where as zoq has no error .

15:18 < ranjan123_> yes

15:18 < zoq> yeah, Ri.insert_rows(3,1); adds another row

15:18 < ranjan123_> data is three-dimensional

15:18 < ranjan123_> coordinate is 4D

15:19 < ranjan123_> sum_over_i (a*x_i + b*y_i + c*t_i + d)^2

15:20 < ranjan123_> coordinate=(a,b,c,d)

15:20 < rcurtin> oh, I see, sorry

15:22 < rcurtin> yeah, I don't see anything wrong here either

15:23 < ranjan123_> hmmm.

15:23 < ranjan123_> :(

15:23 < rcurtin> let me try to run the code on my system

15:24 < rcurtin> *** stack smashing detected ***: ./find_equ terminated

15:24 < rcurtin> looks like there is some invalid memory access

15:25 < ranjan123_> hmm

15:25 < ranjan123_> yes

15:25 < ranjan123_> http://pastebin.com/8CSW6JsZ

15:26 < rcurtin> hang on, I'm digging into it with gdb

15:28 < rcurtin> interesting, if I compile with -fno-stack-protector, it runs fine

15:29 < ranjan123_> yes

15:29 < ranjan123_> it is runs in my system also

15:31 < ranjan123_> valgrind output: http://pastebin.com/0dJMVQuj

15:32 < ranjan123_> Use of uninitialised value of size 8

15:32 < rcurtin> ah now you are onto something

15:32 < rcurtin> I couldn't get valgrind to give me anything useful

15:32 < rcurtin> try valgrind with --track-origins=yes

15:32 < rcurtin> or is it --trace-origins? I can never remember

15:33 < rcurtin> yeah, --track-origins=yes

15:34 < rcurtin> are you sure that your mlpack installation hasn't been modified?

15:34 < ranjan123_> yes

15:34 < ranjan123_> I am sure

15:34 < rcurtin> my guess based on that valgrind output is that Evaluate() is being called with an invalid function index

15:34 < rcurtin> okay

15:35 < rcurtin> can you attach gdb to the process and inspect what the value of i is when this memory issue happens?

15:35 < rcurtin> you can use valgrind and gdb in tandem:

15:35 < ranjan123_> i=24

15:35 < rcurtin> http://billiob.net/blog/20140330_vgdb.html

15:35 < rcurtin> i = 24 causes that issue?

15:35 < rcurtin> but traj.txt has 58 columns

15:36 < ranjan123_> yes

15:36 < rcurtin> so I don't think that I am running the same code you are

15:36 < rcurtin> size_t NumFunctions() const { return 25; }

15:36 < rcurtin> is this different in your setup?

15:36 < ranjan123_> size_t NumFunctions() const { return 55; }

15:36 < rcurtin> (that was 20, but I changed it to 25 to try and reproduce the i=24 error but I could not do it)

15:36 < rcurtin> okay, let me try 55

15:38 < rcurtin> no, I have no issue with 55

15:39 < ranjan123_> ohk

15:39 < rcurtin> ah wait hang on

15:39 < rcurtin> now if I compile without -fno-stack-protector I can reproduce your issue...

15:39 < ranjan123_> for now I can ran it by -fno-stack-protector

15:39 < ranjan123_> ohh

15:40 < ranjan123_> you compiled it with -fno-stack-protector

15:40 < rcurtin> yeah, but needing to compile with -fno-stack-protector is not a great solution in the long run, we should figure out what the actual issue is

15:41 < ranjan123_> hmm. That is right

15:58 < ranjan123_> I guess problem is in armadillo mat .

15:58 < ranjan123_> armadillo_bits/Mat_meat.hpp

16:02 < rcurtin> I very much doubt this is a bug in armadillo

16:03 < rcurtin> when I look through this with gdb, what I can see is that when I try to inspect data1 at the time of the crash, that fails:

16:03 < rcurtin> (gdb) inspect data1

16:03 < rcurtin> Cannot access memory at address 0x8002b1be22ce5ae5

16:03 < rcurtin> but that is a different memory location than during the previous calls to Evaluate()!

16:04 < rcurtin> (where it was 0x7fffffffdfa0)

16:05 < rcurtin> hang on, look at Gradient():

16:05 < rcurtin> if(i==3)

16:05 < rcurtin> {

16:05 < rcurtin> gradient[i]=2*val;

16:05 < rcurtin> }

16:05 < rcurtin> else

16:05 < rcurtin> {

16:05 < rcurtin> gradient[i]=2*val*Ri[i];

16:05 < rcurtin> }

16:05 < ranjan123_> yes

16:05 < rcurtin> but i takes values between 0 and 55, not 0 and 4---and the gradient is of size 4

16:05 < rcurtin> so there is your invalid access

16:06 < ranjan123_> na

16:06 < ranjan123_> coordinate=(a,b,c,d)

16:06 < ranjan123_> so there will be 4 gradient

16:07 < rcurtin> no

16:08 < zoq> nice catch

16:08 < rcurtin> the 'i' parameter to the Gradient() method indicates which of the functions the gradient is being taken for

16:08 < rcurtin> for your SGD situation, 'i' refers to the data point you are looking for

16:08 < ranjan123_> ahhh

16:08 < rcurtin> yeah

16:08 < rcurtin> so I think i is misused in those couple lines

16:08 < rcurtin> maybe you meant to use j within the loop, or something? I am not sure

16:08 < rcurtin> I think that what is happening on our systems is this:

16:09 < rcurtin> the compiler arranges things in memory such that the memory address 26 bytes after the start of the gradient memory is the address of the 'data1' matrix

16:09 < rcurtin> so when i = 26, we end up overwriting the address of the data1 matrix and everything goes to hell

16:09 < ranjan123_> hmmm..

16:09 < rcurtin> but on marcus's system which I think is OS X, the compiler probably does something a little bit different which is why he was not able to replicate the issue

16:10 < ranjan123_> oops

16:10 < rcurtin> OS X or FreeBSD, I don't know what is being used today :)

16:10 < zoq> FreeBSD :)

16:11 < rcurtin> was the compiler clang or gcc, out of curiosity?

16:11 < zoq> clang

16:11 < ranjan123_> clang

16:11 < rcurtin> ranjan123_: you were using clang too?

16:11 < ranjan123_> na

16:12 < ranjan123_> gcc

16:12 < rcurtin> ahh, okay

16:12 < rcurtin> so I guess clang and gcc must be organizing things in memory differently

16:12 < rcurtin> I gotta get lunch, I'll be back later

16:13 < ranjan123_> hmmm

16:13 < ranjan123_> Thanks ! zoq , rcurtin

16:14 < ranjan123_> I have to go for dinner !

18:18 < ranjan123_> I was kind of lost implementing CG and counfued with sgd . :P . Now it is working with sgd, psgd :D

18:19 < ranjan123_> *confused

18:21 < rcurtin> ranjan123_: glad that you got it working. when I merge #611, if you want to add your name to the list of contributors in src/mlpack/core.hpp and COPYRIGHT.txt and open another PR, I'll merge it

18:22 < rcurtin> #611 is maybe a small contribution but it is still a contribution :)

18:22 < ranjan123_> haha

18:23 < ranjan123_> I will add that later

18:23 < ranjan123_> :)

19:12 < kirizaki> Hello everone

19:13 < kirizaki> sorry for long offline

19:13 < kirizaki> some difficulties with new job ;)

19:14 < rcurtin> kirizaki: hi there, long time no see! :)

19:14 < rcurtin> hope the new job is going well :)

19:14 < kirizaki> yes

19:15 < kirizaki> I'm getting little bit closer with IT dept. :P

19:15 < rcurtin> haha

19:16 < rcurtin> it was the same way when I started my new job in August

19:39 < travis-ci> mlpack/mlpack#723 (master - 78f6ac2 : Ryan Curtin): The build was broken.

19:39 < travis-ci> Change view : https://github.com/mlpack/mlpack/compare/460e326cb2dd...78f6ac2e036f

19:39 < travis-ci> Build details : https://travis-ci.org/mlpack/mlpack/builds/122331415