The Bayesian vs frequentist approaches (Part 3) parametric vs non-parametric models

This put up continues our dialogue on the Bayesian vs the frequentist approaches. Here, we ponder implications for parametric and non-parametric models

 

In the sooner weblog the Bayesian vs frequentist approaches: implications for machine learning half two, we talked about that

 

In Bayesian statistics, parameters are assigned an opportunity whereas throughout the frequentist technique, the parameters are mounted. Thus, in frequentist statistics, we take random samples from the inhabitants and goal to find a set of mounted parameters that correspond to the underlying distribution that generated the knowledge. In distinction for Bayesian statistics, we take your full data and goal to look out the parameters of the distribution that generated the knowledge nonetheless we ponder these parameters as probabilities i.e. not mounted.

 

The question then is

How does this dialogue (Bayesian vs frequentist) extend to parametric and non-parametric models?

To recap: In a parametric model, we have a finite number of parameters. In distinction, for nonparametric models, the number of parameters is (most likely) infinite and further notably, the number of parameters and the complexity of the model grows with rising data.

The phrases parametric and non-parametric moreover apply to the underlying distribution. Intuitively, you may probably say that parametric models adjust to a specified distribution – which is printed by the parameters. Non-parametric models do not recommend an underlying distribution.

Another method to technique the problem is to consider algorithms learning a carry out.

You can contemplate machine learning as learning an unknown carry out that maps the inputs X to outputs Y. The primary format of this carry out is Y = f(x). The algorithm learns the kind of the carry out from the teaching data. Different algorithms make completely totally different assumptions or biases with reference to the kind of the carry out and the best way it might be realized. By this technique, parametric machine learning algorithms are algorithms that simplify the carry out to a recognized type. This technique has benefits because of parametric machine learning algorithms are easier, sooner and wish a lot much less data. However, not all unknown options will likely be expressed neatly inside the kind of comparatively simple options like linear options. Hence, parametric models are further fitted to easier points.

In distinction, non-parametric models do not make string assumptions regarding the underlying carry out. Non parametric models take pleasure in flexibility, power and better effectivity. However, as well as they need further data, are slower and liable to overfitting.

To recap, we talked about sooner than, because of the phrases parametric and non-parametric moreover apply to the underlying distribution, you may probably say that parametric models adjust to a specified distribution – which is printed by the parameters. Non-parametric models do not recommend an underlying distribution.

Hence, we’re capable of conclude that whereas the concepts are tangentially related they do not recommend a connection.

Specifically

• Parametric models recommend that the knowledge comes from a recognized distribution and the model can infer the parameters of that distribution
• However, this does not recommend that parametric models are Bayesian (on the grounds that Bayesian models assume a distribution for the parameters)
• To emphasize, Bayesian models relate to prior information.
• As we see increasingly more, the Bayesian vs frequentist concern is further a precedence for statisticians, For machine learning, we’re working often at a greater diploma of abstraction – and totally different considerations apply

The closing, concluding part of this weblog, will carry all these ideas collectively

References

https://machinelearningmastery.com/parametric-and-nonparametric-machine-learning-algorithms/

https://sebastianraschka.com/faq/docs/parametric_vs_nonparametric.html

 

Image provide:nps