19:43 < kris1> For implementation of General Gibbs sampling i could have any n cond. distribution and these cond. distributions will take different number of parameters. How should i go about implementing this any ideas?

19:48 < kris1> The only solution i had was to create a new class called ConditionalDistribution which lets the 1 necessary function double sample(std::vector<double> variables)

19:48 < kris1> is this the correct way ??

19:49 < lozhnikov> Hi Kris. You can just add an activation function

19:52 < kris1> do you mean to say that the conditional distributions would be activation functions

19:52 < kris1> That would be true for the case of the rbm but i was trying to build a more general gibbs sampler. What do you think ?

19:54 < kris1> lozhnikov:

19:56 < lozhnikov> Why do you think that shouldn't work with the general sampler?

19:58 < kris1> Activation functions are not probability distributions right they don't need to integrate to 1??

19:59 < kris1> and in gibbs sampling you require a bunch of conditional probability distributions

19:59 < kris1> so how are these two connected ??

20:01 < lozhnikov> I think activation functions can just sample values with given probability. And LayerType::Farward() can calculate those probabilities

20:05 < kris1> Suppose i use the identity function as the activation function can you explain to me how it would sample from a let's say a normal distribution

20:09 < lozhnikov> I am not sure that I understand your question right. Do you mean that the activation function should know the mean and the variance?

20:12 < kris1> Suppose i have n variables distribution P(x1,x2,.....,xn)[Joint probability]. I would need n conditional distributions right something like this P(x1| x2,x3,....xn) = N(sigma(x_i),1) and similarly for the other variables also.

20:12 < kris1> What i understand from what you are saying is that P(x1|x2,x3,...xn) = ActivationFunction

20:13 < kris1> ActivationFunction(x2,x3,...xn) which is now used to sample the value of x1

20:13 < kris1> is that what you are trying to say ??

20:19 < kris1> Lozhnikov:

20:23 < lozhnikov> I think it is better to pass the mean and the variance to the activation function than variables x1,...xn

20:25 < kris1> But how would the activation function be equivalent to Normal Distribution or Gamma Distribution

20:27 < lozhnikov> you could use std::normal_distribution for sampling 1D samples

20:30 < kris1> Wait i think i should post a gist it would make the doubt more clear.

21:04 < kris1> lozhnikov: have a look at this https://gist.github.com/kris-singh/cdd952a793b7b7dc932ddac83c579761

21:07 < kris1> The question is the there can be multiple distribution for each conditional distribution

21:12 < lozhnikov> Hmm.. If I remember right each conditional distribution is gauss distribution with diagonal covariation matrix. That means all components are independent. So you can sample each component separately

21:13 < lozhnikov> Well.. I think I have to prepare a gist with my vision of the problem. I'll prepare that tomorrow

21:25 < kris1> That is true if the joint distribution is Gaussian then yes you are correct that the conditional will be gaussian

21:25 < kris1> but we cannot make such a assumption in the general case

21:26 < kris1> maybe you could look here

21:26 < kris1> https://darrenjw.wordpress.com/2011/07/16/gibbs-sampler-in-various-languages-revisited/

21:29 < lozhnikov> ok, I'll take a look tomorrow

21:31 < kris1> Thanks.