Document worth reading: “Monte Carlo Gradient Estimation in Machine Learning”

This paper is a broad and accessible survey of the methods we now have at our disposal for Monte Carlo gradient estimation in machine learning and all through the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is built-in; the problem of sensitivity analysis. In machine learning evaluation, this gradient draw back lies on the core of many learning points, in supervised, unsupervised and reinforcement learning. We will often search to rewrite such gradients in a sort that allows for Monte Carlo estimation, allowing them to be merely and successfully used and analysed. We uncover three strategies–the pathwise, ranking function, and measure-valued gradient estimators–exploring their historic developments, derivation, and underlying assumptions. We describe their use in completely different fields, current how they’re related and may very well be blended, and broaden on their attainable generalisations. Wherever Monte Carlo gradient estimators have been derived and deployed in the earlier, very important advances have adopted. A deeper and further widely-held understanding of this draw back will outcome in extra advances, and it is these advances that we need to assist. Monte Carlo Gradient Estimation in Machine Learning