rcurtin_irc changed the topic of #mlpack to: mlpack: a scalable machine learning library (https://www.mlpack.org/) -- channel logs: https://libera.irclog.whitequark.org/mlpack -- NOTE: messages sent here might not be seen by bridged users on matrix, gitter, or slack
<jonpsy[m]> > @jonpsy: https://pytorch.org/docs/stable/generated/torch.nn.Module.html?highlight=forward#torch.nn.Module.forward the note on this explain why the class itself is called instead of the forward method of the class.
<jonpsy[m]> thanks
<jonpsy[m]> so i found out that it is indeed training
<jonpsy[m]> but the error seems to be with the env
<jonpsy[m]> fixed!
<jonpsy[m]> say4n: could you have a look at ```agent.learn()``` code part?
<say4n[m]> jonpsy[m]: Sure, I can have a look later today. Specifically what about it?
<jonpsy[m]> so, i suppose they've used ```model``` and ```model_``` as ```learningNetwork``` and ```targetNetwork```, but im not sure
<jonpsy[m]> and how theyve implemented the algo in general.
<jonpsy[m]> we could have a chat after you've had a pass, thoughts?
<jonpsy[m]> * and how theyve implemented the algo in general. ```loss```
<say4n[m]> okay
<jonpsy[m]> thanks a lot!
rcurtin_1rc has joined #mlpack
AbhishekMishra[m has quit [*.net *.split]
LokeshJawale[m] has quit [*.net *.split]
RudraPatil[m] has quit [*.net *.split]
HrithikNambiar[m has quit [*.net *.split]
jonpsy[m] has quit [*.net *.split]
jjb[m] has quit [*.net *.split]
JatoJoseph[m] has quit [*.net *.split]
_slack_mlpack_34 has quit [*.net *.split]
_slack_mlpack_16 has quit [*.net *.split]
Shadow3049[m] has quit [*.net *.split]
mlpack-inviter[m has quit [*.net *.split]
KrishnaSashank[m has quit [*.net *.split]
Pushker[m] has quit [*.net *.split]
rcurtin[m] has quit [*.net *.split]
AbhinavGudipati[ has quit [*.net *.split]
GauravGhati[m] has quit [*.net *.split]
M7Ain7Soph77Ain7 has quit [*.net *.split]
MayankRaj[m] has quit [*.net *.split]
halfy has quit [*.net *.split]
ShivamShaurya[m] has quit [*.net *.split]
_slack_mlpack_25 has quit [*.net *.split]
_slack_mlpack_13 has quit [*.net *.split]
_slack_mlpack_U7 has quit [*.net *.split]
Cadair has quit [*.net *.split]
MatheusAlcntaraS has quit [*.net *.split]
jeffin143[m] has quit [*.net *.split]
AbhishekNimje[m] has quit [*.net *.split]
VarunGupta[m] has quit [*.net *.split]
AyushiJain[m] has quit [*.net *.split]
ABHINAVANAND[m] has quit [*.net *.split]
dkipke[m] has quit [*.net *.split]
ShivamNayak[m] has quit [*.net *.split]
OleksandrNikolsk has quit [*.net *.split]
DavidportlouisDa has quit [*.net *.split]
RishabhGoel[m] has quit [*.net *.split]
DivyanshKumar[m] has quit [*.net *.split]
Kaushalc64[m] has quit [*.net *.split]
SergioMoralesE[m has quit [*.net *.split]
ServerStatsDisco has quit [*.net *.split]
swaingotnochills has quit [*.net *.split]
AyushSingh[m] has quit [*.net *.split]
SlackIntegration has quit [*.net *.split]
zoq[m]1 has quit [*.net *.split]
GopiManoharTatir has quit [*.net *.split]
sailor[m] has quit [*.net *.split]
NippunSharmaNipp has quit [*.net *.split]
M074AABGKS has quit [*.net *.split]
_slack_mlpack_27 has quit [*.net *.split]
ShahAnwaarKhalid has quit [*.net *.split]
kartikdutt18kart has quit [*.net *.split]
AbdullahKhilji[m has quit [*.net *.split]
_slack_mlpack_14 has quit [*.net *.split]
_slack_mlpack_31 has quit [*.net *.split]
ZanHuang[m] has quit [*.net *.split]
fieryblade[m] has quit [*.net *.split]
AlexNguyen[m] has quit [*.net *.split]
shrit[m] has quit [*.net *.split]
rcurtin_irc has quit [*.net *.split]
AbhishekMishra[m has joined #mlpack
LokeshJawale[m] has joined #mlpack
RudraPatil[m] has joined #mlpack
HrithikNambiar[m has joined #mlpack
JatoJoseph[m] has joined #mlpack
_slack_mlpack_34 has joined #mlpack
jjb[m] has joined #mlpack
jonpsy[m] has joined #mlpack
_slack_mlpack_16 has joined #mlpack
KshitijAggarwal[ has quit [Ping timeout: 240 seconds]
AvikantSrivastav has quit [Ping timeout: 240 seconds]
TrinhNgo[m] has quit [Ping timeout: 240 seconds]
SaiVamsi[m] has quit [Ping timeout: 240 seconds]
MohomedShalik[m] has quit [Ping timeout: 240 seconds]
RishabhGarg108[m has quit [Ping timeout: 240 seconds]
_slack_mlpack_37 has quit [Ping timeout: 240 seconds]
AbhishekMishra[m has quit [Ping timeout: 245 seconds]
LokeshJawale[m] has quit [Ping timeout: 245 seconds]
HrithikNambiar[m has quit [Ping timeout: 245 seconds]
jjb[m] has quit [Ping timeout: 245 seconds]
_slack_mlpack_34 has quit [Ping timeout: 245 seconds]
rcurtin_matrixor has quit [Ping timeout: 252 seconds]
MatrixTravelerbo has quit [Ping timeout: 252 seconds]
HARSHCHAUHAN[m] has quit [Ping timeout: 252 seconds]
prasad-dashprasa has quit [Ping timeout: 252 seconds]
jonathanplatkiew has quit [Ping timeout: 252 seconds]
AmanKashyap[m] has quit [Ping timeout: 252 seconds]
Aakash-kaushikAa has quit [Ping timeout: 252 seconds]
NitikJain[m] has quit [Ping timeout: 252 seconds]
abernauer[m] has quit [Ping timeout: 252 seconds]
VedantaJha[m] has quit [Ping timeout: 252 seconds]
GauravGhati[m] has joined #mlpack
rcurtin[m] has joined #mlpack
shrit[m] has joined #mlpack
MayankRaj[m] has joined #mlpack
KrishnaSashank[m has joined #mlpack
Shadow3049[m] has joined #mlpack
AbhinavGudipati[ has joined #mlpack
M7Ain7Soph77Ain7 has joined #mlpack
mlpack-inviter[m has joined #mlpack
_slack_mlpack_13 has joined #mlpack
Pushker[m] has joined #mlpack
AlexNguyen[m] has joined #mlpack
ShivamShaurya[m] has joined #mlpack
_slack_mlpack_25 has joined #mlpack
_slack_mlpack_U7 has joined #mlpack
AyushSingh[m] has joined #mlpack
M074AABGKS has joined #mlpack
ZanHuang[m] has joined #mlpack
swaingotnochills has joined #mlpack
kartikdutt18kart has joined #mlpack
sailor[m] has joined #mlpack
NippunSharmaNipp has joined #mlpack
fieryblade[m] has joined #mlpack
zoq[m]1 has joined #mlpack
ShahAnwaarKhalid has joined #mlpack
SlackIntegration has joined #mlpack
GopiManoharTatir has joined #mlpack
_slack_mlpack_27 has joined #mlpack
AbdullahKhilji[m has joined #mlpack
_slack_mlpack_14 has joined #mlpack
_slack_mlpack_31 has joined #mlpack
Cadair has joined #mlpack
AyushiJain[m] has joined #mlpack
AbhishekNimje[m] has joined #mlpack
MatheusAlcntaraS has joined #mlpack
jeffin143[m] has joined #mlpack
VarunGupta[m] has joined #mlpack
ABHINAVANAND[m] has joined #mlpack
OleksandrNikolsk has joined #mlpack
RishabhGoel[m] has joined #mlpack
dkipke[m] has joined #mlpack
DavidportlouisDa has joined #mlpack
Kaushalc64[m] has joined #mlpack
ShivamNayak[m] has joined #mlpack
DivyanshKumar[m] has joined #mlpack
SergioMoralesE[m has joined #mlpack
Gulshan[m] has quit [Ping timeout: 240 seconds]
_slack_mlpack_28 has quit [Ping timeout: 240 seconds]
ABoodhayanaSVish has quit [Ping timeout: 240 seconds]
swaingotnochills has quit [Ping timeout: 252 seconds]
AyushSingh[m] has quit [Ping timeout: 252 seconds]
SlackIntegration has quit [Ping timeout: 252 seconds]
zoq[m]1 has quit [Ping timeout: 252 seconds]
sailor[m] has quit [Ping timeout: 252 seconds]
NippunSharmaNipp has quit [Ping timeout: 252 seconds]
GopiManoharTatir has quit [Ping timeout: 252 seconds]
fieryblade[m] has quit [Ping timeout: 252 seconds]
ZanHuang[m] has quit [Ping timeout: 252 seconds]
M074AABGKS has quit [Ping timeout: 252 seconds]
_slack_mlpack_14 has quit [Ping timeout: 252 seconds]
_slack_mlpack_27 has quit [Ping timeout: 252 seconds]
ShahAnwaarKhalid has quit [Ping timeout: 252 seconds]
AbdullahKhilji[m has quit [Ping timeout: 252 seconds]
kartikdutt18kart has quit [Ping timeout: 252 seconds]
_slack_mlpack_31 has quit [Ping timeout: 252 seconds]
ArunavShandeelya has quit [Ping timeout: 252 seconds]
Gman[m] has quit [Ping timeout: 252 seconds]
GaborBakos[m] has quit [Ping timeout: 252 seconds]
KumarArnav[m] has quit [Ping timeout: 252 seconds]
RudraPatil[m] has quit [Ping timeout: 245 seconds]
jonpsy[m] has quit [Ping timeout: 245 seconds]
JatoJoseph[m] has quit [Ping timeout: 245 seconds]
_slack_mlpack_16 has quit [Ping timeout: 245 seconds]
mlpack-inviter[m has quit [Ping timeout: 256 seconds]
Shadow3049[m] has quit [Ping timeout: 256 seconds]
KrishnaSashank[m has quit [Ping timeout: 256 seconds]
AlexNguyen[m] has quit [Ping timeout: 256 seconds]
Pushker[m] has quit [Ping timeout: 256 seconds]
rcurtin[m] has quit [Ping timeout: 256 seconds]
GauravGhati[m] has quit [Ping timeout: 256 seconds]
AbhinavGudipati[ has quit [Ping timeout: 256 seconds]
M7Ain7Soph77Ain7 has quit [Ping timeout: 256 seconds]
MayankRaj[m] has quit [Ping timeout: 256 seconds]
_slack_mlpack_13 has quit [Ping timeout: 256 seconds]
shrit[m] has quit [Ping timeout: 256 seconds]
_slack_mlpack_25 has quit [Ping timeout: 256 seconds]
ShivamShaurya[m] has quit [Ping timeout: 256 seconds]
_slack_mlpack_U7 has quit [Ping timeout: 256 seconds]
jeffin143[m] has quit [Ping timeout: 272 seconds]
MatheusAlcntaraS has quit [Ping timeout: 272 seconds]
Cadair has quit [Ping timeout: 272 seconds]
AbhishekNimje[m] has quit [Ping timeout: 272 seconds]
dkipke[m] has quit [Ping timeout: 272 seconds]
AyushiJain[m] has quit [Ping timeout: 272 seconds]
DavidportlouisDa has quit [Ping timeout: 272 seconds]
OleksandrNikolsk has quit [Ping timeout: 272 seconds]
ABHINAVANAND[m] has quit [Ping timeout: 272 seconds]
VarunGupta[m] has quit [Ping timeout: 272 seconds]
ShivamNayak[m] has quit [Ping timeout: 272 seconds]
SergioMoralesE[m has quit [Ping timeout: 272 seconds]
DivyanshKumar[m] has quit [Ping timeout: 272 seconds]
RishabhGoel[m] has quit [Ping timeout: 272 seconds]
Kaushalc64[m] has quit [Ping timeout: 272 seconds]
HarshVardhanKuma has quit [Ping timeout: 272 seconds]
SoumyadipSarkar[ has quit [Ping timeout: 272 seconds]
swaingotnochill[ has quit [Ping timeout: 272 seconds]
ryan[m]1 has quit [Ping timeout: 272 seconds]
say4n[m] has quit [Ping timeout: 272 seconds]
_slack_mlpack_10 has quit [Ping timeout: 272 seconds]
_slack_mlpack_22 has quit [Ping timeout: 272 seconds]
GauravTirodkar[m has quit [Ping timeout: 272 seconds]
Gauravkumar[m] has quit [Ping timeout: 268 seconds]
sdev_7211[m] has quit [Ping timeout: 268 seconds]
bisakh[m] has quit [Ping timeout: 268 seconds]
ChaithanyaNaik[m has quit [Ping timeout: 268 seconds]
ManishKausikH[m] has quit [Ping timeout: 268 seconds]
ronakypatel[m] has quit [Ping timeout: 268 seconds]
huberspot[m] has quit [Ping timeout: 268 seconds]
FranchisNSaikia[ has quit [Ping timeout: 268 seconds]
Amankumar[m] has quit [Ping timeout: 268 seconds]
Saksham[m] has quit [Ping timeout: 268 seconds]
RishabhGarg108Ri has quit [Ping timeout: 268 seconds]
_slack_mlpack_17 has quit [Ping timeout: 268 seconds]
_slack_mlpack_24 has quit [Ping timeout: 268 seconds]
EricTroupeTester has quit [Ping timeout: 268 seconds]
SiddhantJain[m] has quit [Ping timeout: 276 seconds]
TathagataRaha[m] has quit [Ping timeout: 276 seconds]
HemalMamtora[m] has quit [Ping timeout: 276 seconds]
DillonKipke[m] has quit [Ping timeout: 276 seconds]
DirkEddelbuettel has quit [Ping timeout: 276 seconds]
Prometheus[m] has quit [Ping timeout: 276 seconds]
LolitaNazarov[m] has quit [Ping timeout: 276 seconds]
fazamuhammad[m] has quit [Ping timeout: 276 seconds]
M068AABMUC has quit [Ping timeout: 276 seconds]
gitter-badgerThe has quit [Ping timeout: 276 seconds]
zoq[m] has quit [Ping timeout: 276 seconds]
heisenbuugGopiMT has quit [Ping timeout: 276 seconds]
AyushKumarLavani has quit [Ping timeout: 276 seconds]
_slack_mlpack_U4 has quit [Ping timeout: 276 seconds]
_slack_mlpack_19 has quit [Ping timeout: 276 seconds]
_slack_mlpack_U0 has quit [Ping timeout: 276 seconds]
_slack_mlpack_U0 has joined #mlpack
_slack_mlpack_U4 has joined #mlpack
_slack_mlpack_U7 has joined #mlpack
_slack_mlpack_13 has joined #mlpack
_slack_mlpack_10 has joined #mlpack
_slack_mlpack_34 has joined #mlpack
_slack_mlpack_16 has joined #mlpack
_slack_mlpack_22 has joined #mlpack
_slack_mlpack_37 has joined #mlpack
_slack_mlpack_31 has joined #mlpack
_slack_mlpack_19 has joined #mlpack
_slack_mlpack_25 has joined #mlpack
_slack_mlpack_28 has joined #mlpack
_slack_mlpack_14 has joined #mlpack
LokeshJawale[m] has joined #mlpack
HrithikNambiar[m has joined #mlpack
jjb[m] has joined #mlpack
AbhishekMishra[m has joined #mlpack
AvikantSrivastav has joined #mlpack
RishabhGarg108[m has joined #mlpack
MohomedShalik[m] has joined #mlpack
SaiVamsi[m] has joined #mlpack
KshitijAggarwal[ has joined #mlpack
TrinhNgo[m] has joined #mlpack
Aakash-kaushikAa has joined #mlpack
NitikJain[m] has joined #mlpack
HARSHCHAUHAN[m] has joined #mlpack
abernauer[m] has joined #mlpack
VedantaJha[m] has joined #mlpack
jonathanplatkiew has joined #mlpack
AmanKashyap[m] has joined #mlpack
prasad-dashprasa has joined #mlpack
KumarArnav[m] has joined #mlpack
AlexNguyen[m] has joined #mlpack
Pushker[m] has joined #mlpack
GauravGhati[m] has joined #mlpack
AbhinavGudipati[ has joined #mlpack
mlpack-inviter[m has joined #mlpack
Shadow3049[m] has joined #mlpack
M7Ain7Soph77Ain7 has joined #mlpack
rcurtin[m] has joined #mlpack
ShivamShaurya[m] has joined #mlpack
shrit[m] has joined #mlpack
KrishnaSashank[m has joined #mlpack
MayankRaj[m] has joined #mlpack
_slack_mlpack_24 has joined #mlpack
_slack_mlpack_17 has joined #mlpack
Gulshan[m] has joined #mlpack
ABoodhayanaSVish has joined #mlpack
_slack_mlpack_27 has joined #mlpack
SoumyadipSarkar[ has joined #mlpack
HarshVardhanKuma has joined #mlpack
swaingotnochill[ has joined #mlpack
GauravTirodkar[m has joined #mlpack
say4n[m] has joined #mlpack
ryan[m]1 has joined #mlpack
JatoJoseph[m] has joined #mlpack
RudraPatil[m] has joined #mlpack
ArunavShandeelya has joined #mlpack
Gman[m] has joined #mlpack
jonpsy[m] has joined #mlpack
GaborBakos[m] has joined #mlpack
jeffin143[m] has joined #mlpack
AbhishekNimje[m] has joined #mlpack
dkipke[m] has joined #mlpack
RishabhGoel[m] has joined #mlpack
MatheusAlcntaraS has joined #mlpack
SergioMoralesE[m has joined #mlpack
Kaushalc64[m] has joined #mlpack
AyushiJain[m] has joined #mlpack
DavidportlouisDa has joined #mlpack
DivyanshKumar[m] has joined #mlpack
ShivamNayak[m] has joined #mlpack
ABHINAVANAND[m] has joined #mlpack
VarunGupta[m] has joined #mlpack
OleksandrNikolsk has joined #mlpack
AyushSingh[m] has joined #mlpack
GopiManoharTatir has joined #mlpack
ShahAnwaarKhalid has joined #mlpack
fieryblade[m] has joined #mlpack
NippunSharmaNipp has joined #mlpack
kartikdutt18kart has joined #mlpack
M074AABGKS has joined #mlpack
swaingotnochills has joined #mlpack
ZanHuang[m] has joined #mlpack
zoq[m]1 has joined #mlpack
sailor[m] has joined #mlpack
AbdullahKhilji[m has joined #mlpack
bisakh[m] has joined #mlpack
huberspot[m] has joined #mlpack
RishabhGarg108Ri has joined #mlpack
Gauravkumar[m] has joined #mlpack
ManishKausikH[m] has joined #mlpack
ChaithanyaNaik[m has joined #mlpack
ronakypatel[m] has joined #mlpack
FranchisNSaikia[ has joined #mlpack
Saksham[m] has joined #mlpack
AyushKumarLavani has joined #mlpack
Prometheus[m] has joined #mlpack
fazamuhammad[m] has joined #mlpack
zoq[m] has joined #mlpack
TathagataRaha[m] has joined #mlpack
DillonKipke[m] has joined #mlpack
heisenbuugGopiMT has joined #mlpack
SiddhantJain[m] has joined #mlpack
LolitaNazarov[m] has joined #mlpack
HemalMamtora[m] has joined #mlpack
DirkEddelbuettel has joined #mlpack
M068AABMUC has joined #mlpack
Amankumar[m] has joined #mlpack
sdev_7211[m] has joined #mlpack
EricTroupeTester has joined #mlpack
<kartikdutt18kart> Sure.
rcurtin_matrixor has joined #mlpack
ServerStatsDisco has joined #mlpack
gitter-badgerThe has joined #mlpack
MatrixTravelerbo has joined #mlpack
halfy has joined #mlpack
Cadair has joined #mlpack
SlackIntegration has joined #mlpack
<shrit[m]> heisenbuug (Gopi M Tatiraju): would you send me the link for the meeting
<shrit[m]> ?
<heisenbuugGopiMT> Is mlpack room busy?
<zoq[m]> yes
<heisenbuugGopiMT> Okay...
<shrit[m]> I will join in a couple of seconds
<heisenbuugGopiMT> Yupp
<zoq[m]> room is free now
<say4n[m]> <jonpsy[m]> "we could have a chat after you'v" <- went through the method :)
<jonpsy[m]> heyyy!
<jonpsy[m]> @ me when you're free
<jonpsy[m]> btw, could someone please explain what this means?
<heisenbuugGopiMT> I might try. Are S and A which are in power state and action?
<heisenbuugGopiMT> R must be representing Real Numbers. Can you send which alphabet means what? Or is that what you want to know?
<say4n[m]> jonpsy: now?
<jonpsy[m]> yo
<jonpsy[m]> free now right?
<say4n[m]> Yes
<jonpsy[m]> > I might try. Are S and A which are in power state and action?
<jonpsy[m]> > R must be representing Real Numbers. Can you send which alphabet means what? Or is that what you want to know?
<jonpsy[m]> i'll get back to this
<jonpsy[m]> > Yes
<jonpsy[m]> so before we continue, lemme share some more details ive found
<jonpsy[m]> the paper says they're using a modified version of HER
<jonpsy[m]> aka Hindsight Experience Replay
<jonpsy[m]> so the logic they're going for, is
<jonpsy[m]> * aka Hindsight Experience Replay A.2.3
<jonpsy[m]> per episode, they generate a preference vector from some distribution
<jonpsy[m]> lets call it w_0; this is the preference we're meant to learn. During the stage of learning from replay buffer, for each sample of buffer they generate N_w amount of preference vectors.
<jonpsy[m]> then, they search for w' in N_w, such that utility of w0 * Q =~ w' * Q
<jonpsy[m]> you there?
<say4n[m]> yes
<jonpsy[m]> i guess you understood what i was saying
<jonpsy[m]> ive written it in docs as well, so you can always re -read it
<say4n[m]> 💯 :)
<jonpsy[m]> wanna discuss on ```agent.learn()``` ?
<say4n[m]> okay
<jonpsy[m]> hm, would you like to start..?
<say4n[m]> right so the first thing they check for is the size of replay buffer and then if they have enough samples in it, they go on to produce minibatches for training
<jonpsy[m]> about the ```state_batch```
<jonpsy[m]> i think they're copying the same state right?
<jonpsy[m]> so, if state was [1]
<say4n[m]> lemme look at batchify
<jonpsy[m]> first, unsqueeze would be ``[[1]]```
<jonpsy[m]> * first, unsqueeze would be ```[[1]]```
<say4n[m]> weight_num is a scalar?
<jonpsy[m]> yes, its the number of weight's sampled from distribution
<jonpsy[m]> * yes, its the number of weights sampled from distribution
<say4n[m]> jonpsy[m]: yes, so all x.s elements of the minibatch are unsqueezed with the map and then these are multiplied with weight_num
<jonpsy[m]> yep so we'l get [ [1], [1], [1] .... weight_num number of times ]
<say4n[m]> right
<jonpsy[m]> btw about the ```model``` and ```model_```
<jonpsy[m]> * about the `model` and `model_`??
<say4n[m]> i think they mention in one of the [comments](https://github.com/RunzheYang/MORL/blob/68602002057762f0c5f318ecb9dddf36d0cc91ab/synthetic/crl/envelope/meta.py#L190) that they have a copy such that it does not interfere with the computation of gradients when `optimizer.backward()` is called
<jonpsy[m]> hm, i thought they were learning & target networks
<say4n[m]> you mean y and Q from eq 6?
<say4n[m]> also one more question, do they reassign the values to be the same for model and model_ apart from when they do init?
<jonpsy[m]> i dont think so..
<jonpsy[m]> you mean making them equal, right?
<say4n[m]> yes
<jonpsy[m]> hm if they wouldve done that, then it wouldve been target and learning network , right?
<say4n[m]> one sec
<say4n[m]> okay from the loss function definition in the code, tauQ should be our y
<say4n[m]> * okay from the [loss function definition](https://github.com/RunzheYang/MORL/blob/68602002057762f0c5f318ecb9dddf36d0cc91ab/synthetic/crl/envelope/meta.py#L229) in the code, tauQ should be our y
<jonpsy[m]> yeah the optimality symbol
<jonpsy[m]> wondering about the mask thingy...
<say4n[m]> mask for nonterminal states
<jonpsy[m]> yeah
<jonpsy[m]> btw the paper did mention about usign target + learning network
<jonpsy[m]> and you're idea that its the same network, just detached seems to be correct
<jonpsy[m]> so, im wondering where are the two networks
<say4n[m]> jonpsy[m]: you mean the model definitions?
<say4n[m]> jonpsy[m]: my hypothesis is that they mask and use all but the terminal batches because the terminal minibatch may be smaller than their mini-batch size.
<jonpsy[m]> about this
<say4n[m]> ah that is why they are probably masking it right?
<jonpsy[m]> Yeaaaha
<jonpsy[m]> makes sense right?
<say4n[m]> yess!
<jonpsy[m]> : )
<jonpsy[m]> okay so that part is clar
<say4n[m]> * ~my hypothesis is that they mask and use all but the terminal batches because the terminal minibatch may be smaller than their mini-batch size.~
<say4n[m]> > <@jonpsy:matrix.org> yeah
<jonpsy[m]> > you mean the model definitions?
<jonpsy[m]> and their usage, eyeah
<say4n[m]> * my hypothesis is that they mask and use all but the terminal batches because the terminal minibatch may be smaller than their mini-batch size.
<say4n[m]> > <@jonpsy:matrix.org> > you mean the model definitions?
<say4n[m]> wait
<say4n[m]> >
<say4n[m]> > and their usage, eyeah
<say4n[m]> ^ here they define the `EnvelopeLinearCQN` model
<jonpsy[m]> yeah..
<jonpsy[m]> so thats the model type right?
<jonpsy[m]> i was talkin about ```learningNetwork``` and ```targetNetwork```
<say4n[m]> yes that is the model, the forward propagation is implemented in the forward() method
<say4n[m]> as in the architecture
<jonpsy[m]> yh
<say4n[m]> btw, you were asking about some {16, 32, 64, 32} from the paper right?
<jonpsy[m]> yeah i figured that out
<say4n[m]> 💯
<jonpsy[m]> <jonpsy[m]> "image.png" <- any idea waht this means?
<say4n[m]> left side is probably the Q function parameterised on the preferences
<say4n[m]> I am not entirely sure, it looks something like saying Q_w(s,a) is a subset of a mapping that maps preferences to R^m for all states and actions. In other words, it says that given a preference vector, the Q maps to R^m for each element in SxA.
<say4n[m]> I think James can explain the notation better. 😅
<jonpsy[m]> ah so the SxA power, means for each state actioin pair
<say4n[m]> yes ;)
<jonpsy[m]> wow, first time comin across this notation
<jonpsy[m]> <jonpsy[m]> "i was talkin about ```learningNe" <- thoughts on this?
<say4n[m]> jonpsy[m]: me neither :p
<say4n[m]> jonpsy[m]: none as of now
<say4n[m]> np :)
<jonpsy[m]> Thanks a ton! That's already a lot of help
<shrit[m]> rcurtin: The path for ensmallen is good now, but Cmake is not to extract the version
<shrit[m]> I resolved it 👍️
<shrit[m]> It passed. I think nothing left from my side for this pull request
<rcurtin[m]> 🎉 🎉 🎉
<rcurtin[m]> (I like that the element client displays confetti for a 🎉 😃)
<rcurtin[m]> I am watching the logs---it looks like there are some linking issues?
<rcurtin[m]> ##[error]cf_main.obj(0,0): Error LNK2019: unresolved external symbol wrapper2_dgetrf_ referenced in function "public: static bool __cdecl arma::auxlib::solve_square_rcond<class arma::Mat<double> >(class arma::Mat<double> &,double &,class arma::Mat<double> &,struct arma::Base<double,class arma::Mat<double> > const &,bool)" (??$solve_square_rcond@V?$Mat@N@arma@@@auxlib@arma@@SA_NAEAV?$Mat@N@1@AEAN0AEBU?$Base@NV?$Mat@N@arma@@@1@_N@Z)
<rcurtin[m]> it looks like CMake is not linking against libarmadillo?
<rcurtin[m]> but, I guess, I should wait for the build to finish. I guess we could check `${ARMADILLO_LIBRARIES}` and `${MLPACK_LIBRARIES}` from CMake to see what is being used
<shrit[m]> Well, I did not touch armadillo at all
<shrit[m]> it might be an old error
<rcurtin[m]> yeah---I'll take a look at the build when it's done, and if it did fail, I can take a quick look and get some idea of what might be wrong
<rcurtin[m]> honestly I think getting the ensmallen path right was the hard part here 😃 nothing should be different about Armadillo, so maybe there is some tiny tweak or something but it should not be difficult to get it working