verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
< naywhayare>
the one main remaining thing to do is to display more than one pre-selected visualization on benchmarks.html, but I haven't figured out exactly how to do that right yet...
< naywhayare>
oh. and all the PDF paper links. I guess I'll fix that now...
< zoq>
hm, I liked the color theme form the "old" mlpack history page, now everything is basically yellow, maybe you can change the color of the listings (points)? Like the index and tutorial pages?
< naywhayare>
ah, yeah, I can do that. let me finish with all the links first
< syrius>
been making my way into Windows build territory w/ mlpack. I've been able to get everything built for a dll we're creating for wrapping mlpack
< syrius>
took a while, but had to use the Intel compiler to get everything finally built
< syrius>
the one issue i'm running into now is linking to MKL instead of the lapack/blas libraries that come w/ armadillo is causing a segfault
< naywhayare>
yeah, MSVC (even 2013) still has bugs, it seems
< syrius>
is there any known gotchas with linking to mkl on windows?
< naywhayare>
can you tell me more about how you have Armadillo configured and installed?
< syrius>
yeah, let me find the specifics so i'm giving accurate information
< naywhayare>
Armadillo has this armadillo.lib wrapper to ease linking (so, on Linux, you'd do -larmadillo instead of -llapack -lblas (or -lmkl, etc.))
< naywhayare>
but often this can be more trouble than it's worth on non-standard setups (and I'd say MKL on Windows is nonstandard, at least from the point of view of the Armadillo developers :))
< naywhayare>
so you may be best off unsetting ARMA_USE_WRAPPER in armadillo_bits/config.hpp, and then letting mlpack's CMake try to figure out what to link against... it *should* work right, I think, although I haven't tested with icc and MKL on Windows
< naywhayare>
but anyway, I should wait to respond until you tell me how things are set up :)
< syrius>
ok, for the armadillo config, we're not using the wrapper. we're linking directly to the lapack/blas libraries
< syrius>
there hasn't been any issues linking though, from an actual compile/link perspective
< syrius>
i've enabled c++11 in armadillo config as well
< syrius>
so it's interesting
< syrius>
it's not *totally* broken linking w/ MKL
< syrius>
and i'm building armadillo as a static lib because using BUILD_SHARED_LIBRARY results in a dll only with no lib file
< syrius>
because no functions are exported
< syrius>
naywhayare: it's interesting. Running a K-Means test actually does work, but LARS, ridge regression, naives bayes all crash instantly
< naywhayare>
so if ARMA_USE_WRAPPER is disabled, then there is actually no need for libarmadillo.lib (or whatever it's called); Armadillo is header-only in that case
< syrius>
but the packaged armadillo blas/lapack work w/ all our tests
< syrius>
naywhayare: hmm, ok. i might rebuild mlpack w/ no arma lib to see if it even requires it linking
< naywhayare>
k-means doesn't use any BLAS/LAPACK functionality, but the other functions you mentioned do, which explains the inconsistent crashing behavior
< naywhayare>
if you can provide CMake output for mlpack, that may be helpful in figuring out what's going on
< syrius>
sure, one moment
< naywhayare>
CMake *should* see that ARMA_USE_WRAPPER is disabled and attempt to link manually against LAPACK/BLAS/etc., but that doesn't necessarily mean that it *is* :)
< syrius>
so, one thing about cmake lapack.. i get it to build mlpack.lib and all the examples, but it does finally die on mlpack_test (the very last build item)
< syrius>
right, good point
< syrius>
also, my mkl linking is only providing mkl_rt.dll
< naywhayare>
it dies as in there is a compile error? or a linking error?
< syrius>
that sounds correct, yes? my understanding is that's the correct way of linking it in now
< syrius>
naywhayare: let me check
< naywhayare>
so, I am not a leading world expert on Windows linking, but I think that is correct...? I thought that you need to link against mkl_rt.lib
< naywhayare>
my understanding is (possibly incorrectly) that to link against something on Windows, you have to have the .lib which provides the definitions of what's in the library. but I'm not certain on that, and it's been a while...
< syrius>
naywhayare: oh, you do link against mkl_rt.lib (during link)
< syrius>
but need to provide mkl_rt.dll for the final built dll
< naywhayare>
right
< syrius>
unless i was to statically link in mkl in that case
< naywhayare>
this seems to match my memory (which I'm loading into the cache as we speak... seek times are slow in my head...)
< syrius>
but i don't think that .lib is the right .lib to statically ilnk against, as yuo mention
< syrius>
heh
< syrius>
sorry i'm still trying to find all the cmake logs
< syrius>
i can't find where cmake puts it
< syrius>
i did my building out of the source tree in a 'build' dir under the main mlpack dir
< naywhayare>
hm, it doesn't seem like CMake keeps logs of the output that's shown when the project is configured
< naywhayare>
you could paste the contents of CMakeCache.txt and I can probably reverse-engineer what happened, though
< syrius>
yeah, we'd really like to use it. we're trying to wrap it for use in a major mathematical/statistical programming language package
< syrius>
naywhayare: we are using 1.0.12
< naywhayare>
okay, it looks like the CMakeCache.txt is fine
< syrius>
are there substantial improvements in git master that you recommend we build from?
< naywhayare>
I wouldn't move up to git master quite yet... I think we are about to overhaul some of the API to be more sensible
< naywhayare>
what I was checking was the version of the CMake scripts we have that I'm looking at
< naywhayare>
but I don't think those have changed too significantly since 1.0.12
< naywhayare>
anyway, the next release will be 1.1.0, but we'll provide some sort of compatibility guide or compatibility layer in case we change important parts of the API
< naywhayare>
so looking at the config.hpp, I think I see what's happened
< naywhayare>
ARMA_USE_LAPACK and ARMA_USE_BLAS are commented out, meaning that Armadillo doesn't think it has any BLAS or LAPACK support
< syrius>
hmm
< naywhayare>
so my guess is that the segfault is occuring after mlpack calls some Armadillo function that needs LAPACK/BLAS
< syrius>
ooooh
< syrius>
sorry
< syrius>
i sent you the 'last' version
< naywhayare>
although when that happens it should be printing "solve(): requires LAPACK" or something
< naywhayare>
ah, okay
< syrius>
which was me commenting it out to use the "included" armadillo libs
< syrius>
those were uncommented when using mkl
< syrius>
i apologize. i last worked on this friday before i went out of town. i just got back
< naywhayare>
okay, so ARMA_USE_LAPACK and ARMA_USE_BLAS are uncommented in the version you are using
< naywhayare>
it's okay, no worries :)
< naywhayare>
are the settings of ARMA_BLAS_LONG, ARMA_BLAS_UNDERSCORE, ARMA_BLAS_CAPITALS, etc. correct for MKL? (I don't know exactly what the MKL conventions for their BLAS replacement are)
< syrius>
i'll have to look into that. i also didn't specify ARMA_USE_MKL_MALLOC.. is that recommended when using MKL?
< naywhayare>
I don't know if it's recommended; I don't think it will make a difference, though, assuming you can call MKL BLAS with objects that weren't created with mkl_malloc()
< syrius>
let me do a little more investigative research into this issue. i don't want to take too much of your time. i just wanted to see if i was banging my head against the wall in vain trying to use MKL on Windows w/ this :)
< naywhayare>
so, I think it should definitely be possible
< syrius>
I'll try a few different variations and look into the variables you specified to see if I can come up with a more precise obstacle
< naywhayare>
one place to start might be to write a simple armadillo program, something like
< naywhayare>
mat X;
< naywhayare>
X.randu(100, 100);
< naywhayare>
vec eigval = eig(X);
< naywhayare>
and then try compiling that, linking against MKL, and seeing if it works
< syrius>
alright, i'll do that. I actually did test that but was doing arma::diagmat test
< naywhayare>
if you can get that to work, then mlpack *should* work (assuming mlpack is linking correctly)
< syrius>
since that's where the debugger on the other machine showed it dying
< syrius>
yeah, it's interesting. we aren't getting any loadlibrary issues or anything like that, and can only load our dll with all the previous dll's (including mkl) loaded
< naywhayare>
hm, I'm not sure if diagmat() uses any BLAS functions
< syrius>
quite possibly not, since it wasn't dying
< syrius>
i'll try your test. that seems like a good point to start from
< naywhayare>
I just suggested eig_sym() (sorry, I miswrote the example, should be eig_sym() not eig()) because it should use BLAS
< naywhayare>
if that passes, then something may be wrong in the mlpack configuration scripts, but I'm happy to help you figure out what's gone wrong
< naywhayare>
(and I'm also very happy to learn what went wrong so I can help the next person who has the issue, or try and make CMake do the right thing in the future)
< syrius>
thanks so much for your help. it's much appreciated. i'll hang around and get back to you. :)
< naywhayare>
sure, no problem :)
travis-ci has joined #mlpack
< travis-ci>
mlpack/mlpack#37 (master - 8aa3abb : Marcus Edel): The build passed.
< syrius>
naywhayare: I think you were right on the ARMA_BLAS_LONG_LONG issue. MKL on Win64 has MKL_INT defined as long long. I'm recompiling mlpack with that setting now
< syrius>
to test
< naywhayare>
cool, let me know if it helps
< syrius>
naywhayare: just finished building mlpack.. continuing on the rest of the steps.. not sure if you recall that i mentioned earlier that mlpack.lib builds as well as the binary examples, but fails at the last test item to build
< syrius>
here is the log from the tail end of that
< syrius>
this is the intel compiler on Windows 7 x64
< naywhayare>
hm, that appears to be some failure in the boost unit test framework
< naywhayare>
can you tell if the compiler is trying to link against libboost_unit_test_framework.lib (or whatever the file is in your setup)?
< naywhayare>
in Linux at a shell I might do this by calling 'VERBOSE=1 make'; it may be similar on your setup (maybe?)
< syrius>
yeah, i hadn't explicitly set the boost variable for the lib itself, just the dir
< syrius>
so it was trying to link against the release boost_unit_test_framework-iw-mt-1_57
< syrius>
well it was that, but 'lib' was prefixed, so i just copied and renamed appropriately so it could find it
< syrius>
but that's how i know it was looking to link against that lib
< naywhayare>
so you made libboost_unit_test_framework-iw-mt-1_57.lib -> boost_unit_test_framework-iw-mt-1_57.lib?
< syrius>
yeah
< naywhayare>
the suffix letters indicate the particular options it was compiled with, so I just want to make sure that's correct
< syrius>
yeah i made sure to copy the release version (which is what i set to build mlpack as)
< syrius>
and not the one with 'gd' as part oft he suffix
< naywhayare>
before you made the chance, icc would complain about not being able to find the library?
< syrius>
yeah
< syrius>
but it's done that with the others too
< syrius>
since when i compiled boost i didn't use the shorten names flag
< naywhayare>
okay, so that isn't happening here... it's finding the library, but the library doesn't seem to have the right symbols in it
< syrius>
exactly
< syrius>
no issues w/ the other boost libs that i built though, so maybe i missed something in the boost configuration specific to the unit test framework lib
< syrius>
anyways i'm not really worried about it, since i do get the lib generated + the example binaries, but from a straight-forward windows perspective w/ the intel compiler, that's the only 'error' i'm running into in the build process
< naywhayare>
yeah, it is definitely something to look into
< naywhayare>
the couple of mailing list posts I can find about it are suggesting that those specific errors surface when there is a version mismatch between the .lib and what the program's compilation options are
< naywhayare>
I don't have the resources to actually dig into and debug this (since I can't reproduce it), but at the very least here are the mailing list threads I uncovered:
< syrius>
alright, i'll take a look. i figured that since the other boost libs (random/program_options) linked fine it might have just been somethign specific to windows. mlpack doesn't seem to be compiled on windows very much from what i've seen so far
< naywhayare>
yeah, Windows isn't focused on too much; we have Windows build servers but I can never find the time to keep them running right
< naywhayare>
someone contributed a simple script to make a redistributable for mlpack, but I haven't had time to build it yet
< naywhayare>
I generally try and make sure it builds on Windows before each release, but often I find that some Armadillo code simply breaks MSVC
< naywhayare>
(while still being valid C++)
< syrius>
yeah
< syrius>
template overloading broke msvc when trying mlpack, so i've been using intel
< naywhayare>
yeah; some people use MinGW
< naywhayare>
but I think the intel compiler will probably produce faster code in the end
< syrius>
hmm, ok, so lars.exe creates the output file fine w/ the sample test data, but the simple test in a DLL dies on the arma::eig_sym call
< syrius>
i need to look at the linking between these two... they are 2 separate machines, so i'm not sure if there's a difference there
< syrius>
but i'm going to create a small sample binary on the build machine instead of trying the dll on the second
< syrius>
with that same eig_sym test
< naywhayare>
sounds to me like mlpack is linking correctly but the eig_sym test isn't, maybe?
< syrius>
hmm... all i did was change the dll to a .exe on the build machine and did the same eig_sym code
< syrius>
and it ran correctly
< syrius>
so the linking must be the same. the problem arises when taking that dll and calling the method on another machine. i'm wondering if lars.exe example would suffer the same fate on that second machine
< syrius>
oh damn
< syrius>
and doing that reveals i think the problem
< syrius>
was never getting any load errors trying the dll method
< syrius>
i didn't know what additional libs mkl_rt.dll was dependent on since they are using LoadLibrary to load them as opposed to directly linking
< syrius>
so nothing was showing up in dependency walker.. uhg
< syrius>
I suck
< syrius>
and even debugging in visual studio never showed any loadlibrary errors
< naywhayare>
huh, fascinating, so is this what was produced the segmentation fault?
< naywhayare>
sorry, that sentence didn't parse... remove the "was" :)
< syrius>
naywhayare: yeah.. there could have been some additional issues.. this is the process of loading a 3rd party dll from a different app
< syrius>
the dll being the one i jsut created.. but because my dll (inside mkl_rt), does a loadlibrary.. it basically didn't get caught and then died
prafiny has joined #mlpack
< prafiny>
Hi ! Anybody there?
< prafiny>
Can I ask for help compiling mlpack on visual studio ? :(
< syrius>
prafiny: I've been doing that last week for a few days. I was only able to successfully compile mlpack using the Intel C++ compiler on Windows
< syrius>
I tried w/ MSVC 2012 & MSVC 2013 and it didn't work
< zoq>
prafiny: MinGW should also work.
< prafiny>
Thanks, incredible all these token errors xD
< prafiny>
I will try with mingw !
< naywhayare>
hi prafiny, the Visual Studio compiler crashes while compiling mlpack
< naywhayare>
its C++11 implementation is incorrect...
< naywhayare>
it's been this way for some years, but maybe VS2016 (or whatever the next version is) will be better
< prafiny>
Hm, too bad this simple task is such a pain :/
< stephentu>
naywhayare: whats new in the last few days
< naywhayare>
syrius: thanks. still a couple of things to do, but I think it's about ready for deployment. I became aware that there isn't a good way on the current site to figure out what mlpack even does
< stephentu>
cool
< stephentu>
i like how it isn't using twitter bootstrap
< stephentu>
like every other site
< naywhayare>
haha, I joked with some friends about making a site like that
prafiny has quit [Ping timeout: 245 seconds]
< syrius>
hmm, naywhayare: the language I'm wrapping mlpack for is very similar to MATLAB. I'm looking for the code that does that to see if it can be modified to work with our language
< syrius>
The only reference I see to MATLAB_BINDINGS in any files is in the CMakeLists.txt and the CMakeCache
< naywhayare>
syrius: in the old days, we had someone who wrote some MATLAB bindings, but they fell into disrepair when he left (and I'm actually not even sure they worked in the first place)
< naywhayare>
let me find a link to what he wrote somewhere in the git history...
< syrius>
ah alright
< naywhayare>
oh, wow, I didn't realize they were still in master. they get removed from every release though, because they don't work:
< naywhayare>
we've had an open GSoC project for a couple of years now on building an automatic binding system, but we have yet to find a student
< naywhayare>
I think that I will devote some time to it once I get a couple of other projects off the plate, because yeah, bindings to other languages are pretty important
< naywhayare>
perhaps not everyone wants to deal with the nightmare that is C++? I don't understand why :)
< syrius>
haha yeah
< syrius>
i see you guys have the R project that looks pretty current
< naywhayare>
yeah, that's maintained by Qiang Kou; he keeps those bindings up to date
< naywhayare>
I personally haven't used them, though
travis-ci has joined #mlpack
< travis-ci>
mlpack/mlpack#38 (master - 648651e : Stephen Tu): The build passed.
< stephentu>
which went away when i took the newline out
< naywhayare>
I have seen something like this before on some arcane system I never bothered to debug further
< naywhayare>
there was a newline somewhere in the CXXFLAGS or something like this, and CMake would just pass it on to the Makefile
< naywhayare>
and GNU make doesn't handle that too nicely...
< naywhayare>
I wonder what specific system configuration makes stephen's not work and marcus's work... maybe CMake version?
< zoq>
hm, okay, I used cmake 3.2.1, 3.1.3 and 2.8.11
< stephentu>
i'm using 3.0.1
< naywhayare>
I'd be surprised if the bug existed in 3.0 but not 2.8, but I suppose it's possible
< stephentu>
well i was actualy using the weird anaconda packaged version
< stephentu>
i've removed it and now am using the brew version
< stephentu>
so maybe i wont run into these problems again
< stephentu>
anyways i may try to hack on LDA on the plane
< stephentu>
battery will probably die before i get very far but
< naywhayare>
my laptop kicked the bucket at the GSoC mentor last year, so I ended up buying a chromebook... the battery on those things goes forever
< naywhayare>
but then again, I was used to 30-40 minute battery life on my old thinkpad :)
< stephentu>
its 5 hrs to the bay area so
< naywhayare>
that said, when I'm using 4 cores to compile on the chromebook, battery life is reduced somewhat
< stephentu>
any laptop that lasts 5 hrs
< stephentu>
is pretty impressive
< naywhayare>
yeah, I think the chromebook I have (a samsung chromebook 2) will last for 8 or 9 hours, if I'm not mistaken
prafiny has joined #mlpack
< prafiny>
Well mingw seems to go full retard on circular include
< syrius>
prafiny: I haven't used MinGW, but have been able to build with Intel's C++ compiler on Windows
< syrius>
unless someone here has a solution for MinGW
< syrius>
which seems (from what I can tell) to be more common than using icl
< prafiny>
I wanted to have a try with mingw for affordability matter. If I read well, icl is not freely available right ?
< syrius>
prafiny: it's not... sorry, I didn't realize that. I'm using it at my company here. Don't usually do the Windows dev and didn't realize it wasn't free
< syrius>
apologies
< prafiny>
It is ok :). It is also for a company but I am a student. So my research budget is a bit like twiddling zero xD
< prafiny>
syrius: Hopefully there is a 30-days evaluation version
< naywhayare>
prafiny: I actually just signed up to get a version of icc... it's free if you're doing academic research, or are a student, or are using it for open-source code
< naywhayare>
they give you a one-year non-commercial license
< naywhayare>
I filled out the form, but they haven't gotten back to me with the link to download yet...
< syrius>
yeah, there were a few things i had to change to get it to work
< syrius>
on Windows i had to add a define NOMINMAX in order to get it to build because of std::max references
< syrius>
on my cmake line i also fill in all the variables directly (for the most part)
< syrius>
i also ended up modifying the included ARMA_FindMKL.cmake file to work w/ latest MKL
< syrius>
I'm also now statically linking in MKL so my change was slightly more intrusive
< prafiny>
naywhayare, could you tell me where I can find this ?
< prafiny>
syrius, in order to make it work with icc then ?
< syrius>
yeah, but if you're not doing MKL then forget the last part i said
< syrius>
I'm compiling now with /Zc:wchar_t to see if it fixes the test binary
< naywhayare>
prafiny: I dunno how long it takes them to get back with the download... I've been waiting probably 6 hours now (so I guess it's not automated)
< naywhayare>
syrius: oh boy, that one sounds like a lot of fun... here's hoping the simple fix works :)
< syrius>
for sure, I'll let you know. it's about 50%
< syrius>
soooo not a fan of windows dev environment.. building on osx was so much easier, and i'm a linux guy, so i'll take that any day.
< naywhayare>
yeah, this is another reason why the Windows build is not very polished; setting up the environment is a disaster
< naywhayare>
and once you have it set up, six months later it's completely out of date and you basically have to start over...
< syrius>
we're just getting this up on windows for our actual language application developer can test the algo's from our product.. i suspect most people doing serious machine learning aren't going to be doing it on a windows box though, heh
< naywhayare>
I hear a surprising amount about it on the mailing list, but I suspect part of that is that most people using it on linux have no need to ask questions on the mailing list since it's so much easier
< syrius>
oh yeah, absolutely
< syrius>
one thign that did bite us hardcore on osx was libstdc++ vs libc++
< syrius>
abi incompatibility was segfaulting on std::vector deconstructor