verne.freenode.net changed the topic of #mlpack to: http://www.mlpack.org/ -- We don't respond instantly... but we will respond. Give it a few minutes. Or hours. -- Channel logs: http://www.mlpack.org/irc/
travis-ci has joined #mlpack
< travis-ci>
mlpack/mlpack#1046 (master - 4129a7c : Ryan Curtin): The build passed.
< marcosirc>
you have to select the view: "Metric analysis with multiple parameters for an algorithm/dataset combination"
< sumedhghaisas>
@marcosirc So base cases are decreasing as we predicted...
< marcosirc>
Yes
< marcosirc>
You can also see the runtime with:
< marcosirc>
method: ALLKNN
< marcosirc>
dataset: isolet
< marcosirc>
option: e
< marcosirc>
params: k:3 seed:42
< marcosirc>
metric: Runtime
< marcosirc>
it will show runtime for flann and ann too.
< marcosirc>
Of course we should do benchmarking with bigger datasets!
< sumedhghaisas>
yeah... I started with cloud dataset... mlpack is slowest there...
< marcosirc>
isolet and corel-histogram are the biggest there..
< sumedhghaisas>
mlpack seems to perform better as the dataset size grows :)
< sumedhghaisas>
which is a very good thing
< sumedhghaisas>
I mean perform relatively better...
< marcosirc>
Yes, I agree.
< marcosirc>
Maybe, we should do some test with bigger datasets...
< sumedhghaisas>
Also the graph for isolet dataset... I cannot see a unit for the y-axis... where as for cloud it states minutes...
< sumedhghaisas>
is minutes common??
< marcosirc>
I think you are confussing the basecases metric with the runtime metric...
< marcosirc>
Ahh no sorry, I see what you mean
< marcosirc>
I don't think m means "minutes". I will check the javascript code to see how it sets the units.
< marcosirc>
I am working updating that view. Now you have to set the same parameters for all libraries. I was planning to modify it.
< marcosirc>
I was planning to use a similar approach to : "Dataset metric plots for any algorithm/parameter combination"
< marcosirc>
where you can add as much cases as you want with different parameters.
< marcosirc>
so we could compare, for example, "mlpack-knn ... -k 3 --seed 42 -t cover" against "mlpack-knn ... -k 3 --seed 42 -t spill" etc
mentekid has quit [Ping timeout: 276 seconds]
< marcosirc>
Would you agree?
< sumedhghaisas>
ohh you mean comparing against trees...
tsathoggua has quit [Quit: Konversation terminated!]
< marcosirc>
I mean, the objeive of this new view is: "show the metric progress for a specific method configuration, with different values for a specific parameter"
< marcosirc>
Now, we compare, for example, the runtime for ALLKNN with the same configuration for different libraries, and different values for the "-e" parameter.
< marcosirc>
I would like to modify the view, so we can set different parameters and see what happen, in each case, when we only change a specific parameter, like "-e".
< marcosirc>
Once we implemented spill trees. We will want to compare, not only mlpack against flann and ann, but also "mlpack -t kd" against "mlpack -t spill".
< marcosirc>
Do you see what I mean?
< sumedhghaisas>
I like the idea... but I think it will get complicated in this process....
< sumedhghaisas>
do you think we should implement it in the same view??
< sumedhghaisas>
or create another view??
< sumedhghaisas>
let me think...
< marcosirc>
I was implemented this in the same view...
< sumedhghaisas>
hmm... I don't see any other way to compare this though...
< marcosirc>
This is the general idea:
< marcosirc>
Initially you choose: a method, a dataset, and an option. (For example: ALKNN, isolet, "-e")
< marcosirc>
Then you can start adding combinations of: parameters, library. As many as you want.
< marcosirc>
For example:
< marcosirc>
"-k 3 -seed 42 -t spill", mlpack
< marcosirc>
"-k 3 -seed 42 -t kd", mlpack
< marcosirc>
"-k 3 -seed 42", flann
< marcosirc>
"-k 3 -seed 42", ann
< marcosirc>
when you click "Redraw graph", all of this cases are shown in the graphic, for different values of "-e".
< sumedhghaisas>
So first parameter then library then combinations.... then add...
< sumedhghaisas>
I mean...
< sumedhghaisas>
in "Dataset metric plots for any algorithm/parameter combination"...
< sumedhghaisas>
we first select method then combination then library...
< sumedhghaisas>
but I don't think tree option will be valid for library...
< sumedhghaisas>
so we can do library first then populate the combinations...
< sumedhghaisas>
what do you think??
< marcosirc>
Yes, I agree.
< marcosirc>
Now, in "Dataset metric plots for any algorithm/parameter combination", when you select parameters, it updates the list of libraries available.
< marcosirc>
So, when you select parameters like: "-k 3 -seed 42 -t spill", only mlpack will appear as a possible library.
< marcosirc>
But, we can move the library to be chosen first, as you suggested. It looks more intuitive...
< sumedhghaisas>
yeah... I think its more intuitive... So if I understand correctly... we will select the method first... then comparison parameter then library... then combination... correct??
< marcosirc>
We could have fixed options: method, dataset, metric, option (For example ALLKNN, isolet, Runtime, -e)
< marcosirc>
Then, users can add as many (library,parameters) combinations as they want.
< nilay>
zoq: Hi, how are you?
< nilay>
I had a small doubt, when doing 1 x 1 convolution we simply ignore some cells of the input tensor?
< sumedhghaisas>
But if we are comparing for different parameter values... shouldn't we keep the method constant??
< sumedhghaisas>
I mean both method and comparison parameter....?
< sumedhghaisas>
ohh you mean the same...
< marcosirc>
Yeah :)
< sumedhghaisas>
sorry :)
< sumedhghaisas>
I think thats a good plan...
< marcosirc>
Great. Thanks! I am working on this. I think I will finish it in the next days.
< zoq>
nilay: Hello, I can't complain, how are you? I'm not sure why do you think we ignore cells of the input tensor? The 1x1 convolution is used to reduce the filter size.
< nilay>
convolution is applying a filter on an input tensor.
< nilay>
so 1 x 1 convolution reduces filter size, what does this mean?
< nilay>
should i read the network in network paper too?
< zoq>
nilay: yeah, that would be helpful
< zoq>
nilay: the 1x1 convolution is a really neat trick, and it's so simple, "For example, an image of 200 x 200 with 50 features on convolution with 20 filters of 1x1 would result in size of 200 x 200 x 20."
< zoq>
nilay: so if you do convolution with a 3x3 kernel you don't have to do it on the 200x200x50 input matrix, just on the 200x200x20 matrix
< nilay>
so in a sense, we are applying this kernel of size 20 x 20 (20 filters of 1 x 1) on the input image which is 200 x 200 x 50
< nilay>
i think i am wrong
< zoq>
no, you are right you do convolution wit a kernel size of 1x1
< nilay>
from where does then 20 filters come? ( a convolution with kernel size 1 x 1 would just be multiplying the image with a scalar)
< nilay>
we implemented the ConvTriangle function, that is convolution right, with a kernel of size k x k
< nilay>
(while doing the feature extraction part)
< nilay>
ok so now i understand i think, we take a tensor at each location which is of size 50, and map it to a tensor of size 20
< nilay>
using some kernell
< nilay>
kernel*
< zoq>
So if you start with an image of size 200x200x20 and you do convolution with a 1x1 filter the output is 200x200 because you accumulate the result over the dimensions of the input.
< zoq>
so, someting like image[0] * kernel + image[1] * kernel ...
< nilay>
yeah, ok. so we only consider the location (i, j) when quoting the convolution size, the tensor at that location is of any arbitrary length
< zoq>
yes, the output of the convolution is defined by the number of kernels
< nilay>
the animation on the link, is not representative then, it seems we are ignoring some values in that link
< nilay>
also the inception layer a - e should be made as separate layers?
< nilay>
(a) layer means 1 x 1 convolution, (b) means a 3 x 3 on top of 1 x 1 and so on, right?
< zoq>
just as a single layer
< zoq>
no need to implement the naive version
< zoq>
unless, you like to do that
< nilay>
is my understanding of what (a) (b) .. mean w.r.t to inception layer correct?
< nilay>
because i don't think it is
< zoq>
I'm lloking at page 5 from the going deeper with convolutions paper
< nilay>
yeah
< nilay>
so they talk about 4a to 4d modules
< nilay>
what is 4a?
< zoq>
let's see
< nilay>
inception (4a)
< zoq>
ah, 4a is a different auxiliary network attached to the overall network
< nilay>
so the inception layer is one unit, and there are auxillary classifiers attached over it in (a - e)
< zoq>
the inception layer is what you see in figure b on page 5, but the overall inception network has some auxillary classifier attached to it
< nilay>
yeah , ok
< zoq>
page 7 figure 3 shows the complete inception network, with two auxillary classifiers attached to it
< zoq>
the output of each, classifier is a softmax activation layer
< nilay>
i wonder why we have a - e when we only attach a softmax classifier at the filter concatenation stage in that figure
Lenish is now known as Rodya
< zoq>
The table on page 6 isn't what figure 3 shows
< zoq>
if you add another inception layer, you probably also add another auxillary classifier
< zoq>
out for dinner, back later
< nilay>
ok
< sumedhghaisas>
@rcurtin: Hey Ryan...
< sumedhghaisas>
our tests are solid... we should boast our coverage... :P
< sumedhghaisas>
I mean on the github readme...
< rcurtin>
okay, if you want to modify the readme do it :)
< sumedhghaisas>
@rcurtin we will need to add --coverage to gcc while compiling... I can try adding a cmake build for it and generate a graphical view using icov ... what do you think?