verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
sumedhghaisas_ has quit [Ping timeout: 260 seconds]
< lozhnikov>
kris1: what is the difference between your implementation of the cd-k algorithm and the the SGD optimizer?
< kris1>
Well for one sgd has no gibbs sampling.
< kris1>
But cd-k has call to gradients function which compute the the gibbs sampling
< lozhnikov>
sgd does the same
< kris1>
You mean implementation wise what’s the diffrence
< lozhnikov>
yeah
< lozhnikov>
your implementation of the cdk optimizer is a subset of the sgd optimizer
< kris1>
Okay so actually there is no need for the evaluate function in case of the cd-k algorithm. Though i am working on one with pseudo-likelihood implemented
< kris1>
The evaluate function is the stopping criteria for the sgd but that is not the case for the cd-k algorithm
< lozhnikov>
Just call the pseudo-likelihood function Evalueate()
< kris1>
I am doing that but it’s not a measure of stopping criteria according to me
< sgupta>
rcurtin: hey Ryan. So, I am compiling all versions of armadillo from different sources. Do I have to take only the major versions (4.000, 4.200,4.300,4.400 etc) or I have to take the 4.320, 4.450 (such versions) as well?
< rcurtin>
hmm, I think if we take too many then we will have too many types in the build matrix
< rcurtin>
the build process for armadillo hasn't changed between versions, so it should be easy to write a nice script that can produce those
< rcurtin>
previously I'd built one for the newest minor releases, so e.g. 4.300.9, 4.400.1, etc.
< sgupta>
rcurtin: I'll just do for all releases. When the process is automated it will not even matter
< rcurtin>
sounds good
< rcurtin>
let me know if there are versions of armadillo you can't find .tar.gzs for, I might have a few I can dig up here and there
< sgupta>
One more suggestion is to create an FTP or a directory on the server with all the tar balls
< sgupta>
So that we have all of them at a single place
< rcurtin>
on masterblaster? we can just put them in /opt/armadillo/ or something like this
< rcurtin>
Conrad intentionally removes old versions, I think to reduce his support load for older versions, so I don't want to "go around him" and post them again if he doesn't want them posted
sumedhghaisas_ has joined #mlpack
< sgupta>
rcurtin: well my idea was to put it in www directory. But yes, as you explained that's not an option
< rcurtin>
hmmm...
< rcurtin>
now that I think about it further, the Docker images may be running on other hosts
< rcurtin>
so local filesystem may not be the way to go
< rcurtin>
how about this: we can put it in a www directory, but only make it accessible to known mlpack build hosts
abhisuma has joined #mlpack
< rcurtin>
I guess we would need to set up apache on masterblaster and add the necessary iptables rules, would you be willing to do that?
abhisuma has quit [Client Quit]
< sgupta>
Yes, but you have to guide me adding those iptable rules
< rcurtin>
sure, I can help with that, iptables is not too scary :)
< rcurtin>
actually, I'll revise that---iptables is definitely scary, but it's not actually that hard :)
< rcurtin>
I added you to the sudo group on masterblaster, so now you should be able to install apache or whatever webserver you'd like to use; I guess lighttpd could work too
< sgupta>
rcurtin: thank you for making me a sudoer
< rcurtin>
sure, please don't type 'sudo halt' :)
< sgupta>
rcurtin: this will make much of my work easy
< rcurtin>
yeah, I realized that it was kind of a hindrance to have you go through me for everything
< rcurtin>
although, I will ask, if you can keep me up to date with the configuration changes you're making so that we can be on the same pages :)
< sgupta>
Sure :)
< sumedhghaisas_>
zoq: Hey Marcus... Wanted to talk to you about the RNN changes to support variable length. I think I have an idea in mind. Just wanted to run it by you first.
< zoq>
sumedhghais: Sure go ahead :)
< sumedhghaisas_>
So for removing the confusing lets first name the RNN 'rho' ... 'seqLength'?
< zoq>
okay
< sumedhghaisas_>
okay. So I was thinking that we can implement a new function that takes a field and iterates over it just like FNN. Although I have a little bit confusion about the LSTM 'rho'. When you said it can be different... you mean it can be different for different inpus?
< sumedhghaisas_>
*inputs
< sumedhghaisas_>
I think it will be same for 1 Training... right?
< zoq>
different from the seqLength and yeah between iterations
< zoq>
so, yeah it's the same for 1 training
< sumedhghaisas_>
so if I am getting this right.. 'seqLength' will be different but 'rho' will be same.
< zoq>
correct
< sumedhghaisas_>
so 'seqLength' can be detected inside the Train function... if given inputSize. LSTM 'rho' will be set by the user.
< sumedhghaisas_>
now the problem that I see is resetting the variables in LSTM when next sequence starts
< zoq>
yes, same for the labels/reponses
< sumedhghaisas_>
as 'rho' might now be the end of sequence
< zoq>
right
< zoq>
you have to somehow propagate that information
< zoq>
we could introduce a new parameter
< sumedhghaisas_>
yup... so ResetVisitor for RNN ? we have to overload the function call for RNN cells as not all cells might have Reset function
< sumedhghaisas_>
sorry not all layers I meant
< sumedhghaisas_>
new parameter? to mark the end of sequence?
< zoq>
I meant new function, so Reset function might be an option
< sumedhghaisas_>
yeah... I was also thinking about marking the end of sequence with a dummy all zeros input
< sumedhghaisas_>
but that seems more complicated.
< zoq>
that would mean we do this in every iteration
< sumedhghaisas_>
yeah... I also thought that inefficient
< zoq>
yeah, I like the Reset idea
< zoq>
sounds like a good plan to me
< sumedhghaisas_>
okay. on it... So for that Evaluate issue... I see what you mean. I totally forgot that point. Sorry
< sumedhghaisas_>
and btw... for variable sequence I have already implemented the list implementation.
< zoq>
I think we could check if the numFunctions = 1 right, and do Evaluate even if currentCost != -1
< zoq>
Would that solve the problem?
< zoq>
If numfunctions > 1 the objective should be okay
< zoq>
What I like about the idea, is that we can avoid another pass through the network just to get the updated objective, we could get at the next iteration for free.
< sumedhghaisas_>
I think so. But what I don't understand is... why gradient checks are failing?
< zoq>
because we do Evaluate that returns the objective and afterwards we do Gradient which returns the same objective as the first one, and since the test expects that the objective changes it fails.
< zoq>
We could update the test, but if someone is going to train with a single iterations the ouutput might be unexpected.
< sumedhghaisas_>
ahhh... okay.
< sumedhghaisas_>
I think your solution should solve the problem
< sumedhghaisas_>
butlet me check again real quick
< sumedhghaisas_>
ahhh wait... I have another idea. My method to solve this is all wrong.
< sumedhghaisas_>
we should keep the Evaluate called from the Optimizer the real evaluate
< sumedhghaisas_>
and call the dummy in Gradient
< sumedhghaisas_>
wait... no
< sumedhghaisas_>
thats wrong
< sumedhghaisas_>
forget I said that :)
< zoq>
hm, check if we call evaluate multiple times :)
< sumedhghaisas_>
okay... your solution definately works :)
< zoq>
I see the last step that calculates the overallobjective over the dataset is problematic.
< zoq>
yeah, we should definitely go for it :)
< sumedhghaisas_>
yeah... adding the check for function index might save us there. what you think?
< zoq>
hm, it was meant as a joke :)
< zoq>
but yeah it should work
< sumedhghaisas_>
I know :P
< sumedhghaisas_>
I got that little late...
< sumedhghaisas_>
btw... just on a side note... how do I use RNN network if I am interested only the last output?
< zoq>
set single = true
< zoq>
I have to step out, back later.
< sumedhghaisas_>
ahh I see.
sgupta has quit [Ping timeout: 246 seconds]
sgupta has joined #mlpack
kris1 has quit [Quit: kris1]
kris1 has joined #mlpack
vivekp has quit [Ping timeout: 268 seconds]
vivekp has joined #mlpack
sumedhghaisas_ has quit [Ping timeout: 260 seconds]
kris1 has quit [Quit: kris1]
pretorium[m] has quit [Ping timeout: 240 seconds]
govg has quit [Ping timeout: 260 seconds]
govg has joined #mlpack
govg has quit [Ping timeout: 240 seconds]
govg has joined #mlpack
sgupta has quit [Ping timeout: 240 seconds]
sgupta has joined #mlpack
sgupta has quit [Ping timeout: 255 seconds]
travis-ci has joined #mlpack
< travis-ci>
mlpack/mlpack#2584 (master - 41cbc50 : Marcus Edel): The build was broken.
mikeling has quit [Quit: Connection closed for inactivity]
kris1 has joined #mlpack
benchmark has joined #mlpack
benchmark has quit [Client Quit]
pretorium[m] has joined #mlpack
sumedhghaisas_ has joined #mlpack
aashay has quit [Quit: Connection closed for inactivity]
< rcurtin>
ironstark: keep me updated if there is anything I can help with with the shogun install
< rcurtin>
I don't use anaconda, so I am not too familiar with what the exact issue might be
< rcurtin>
but from a high level glance it looked like whatever was being compiled needed to link against -lgomp but wasn't doing so; I am not sure why, maybe a CMake bug
< rcurtin>
another option is to get you a shell account on the benchmarking systems, so that you can use the same environment there, but we have to be careful that you don't run any intensive builds or anything on any slaves that are running benchmarks,
< rcurtin>
otherwise the results could be affected
< rcurtin>
sumedhghaisas_: do you want to post a blog entry for week 2? :)
< rcurtin>
ironstark: same question for you, I haven't seen yours this week either :)
vivekp has quit [Ping timeout: 268 seconds]
sgupta has joined #mlpack
< sumedhghaisas_>
rcurtin: ahh yes... I will do that later today. Sorry for that :)
< rcurtin>
no worries, I find the updates really useful to understand what is going on :)
sumedhghaisas has joined #mlpack
< rcurtin>
it helps a lot for me to read those and get the right context to review a PR too
< sumedhghaisas_>
rcurtin: I understand. I have made the changes in the ANN PR. I will update that now. I am knee deep in completing the support for variable length encoding in RNN. Will complete that and write the post.
< sumedhghaisas_>
although technically I am suppose to complete BatchNorm .... but I will do that tomorrow. RNN's are much more fun... :)
< sumedhghaisas_>
I will still complete BatchNorm at the end of the week
kris1 has quit [Quit: kris1]
< rcurtin>
sumedhghaisas_: sure, it's ok, things don't always go exactly according to plan :)