[DOC] two notebooks on Bayesian probabilistic regression #520
Conversation
meraldoantonio
requested review from
felipeangelimvieira,
fkiraly and
SaiRevanth25
as code owners
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
Hi @fkiraly, This PR includes the first two example notebooks for the Bayesian estimators. I suggest we merge these first, and I can submit the remaining notebooks, which are still in progress, in a separate PR. I haven’t made any major changes to these notebooks since your review in the old PR (#500). Let me know your thoughts or if I need to make any change to this PR, thanks! |
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "This series of notebooks offers an in-depth exploration of the **Bayesian Linear Regression**. \n", |
| "Linear regression is a widely used model due to its simplicity and interpretability. \n", | ||
| "\n", | ||
| "\n", | ||
| "In its simplest form, it predicts a single target $t$ as the deterministic output of the function $y$, which in turn is a linear combination of input variables $\\mathbf{x} = (x_1, \\dots, x_D)^\\top$ and parameters $\\mathbf{w} = (w_0, w_1, \\dots, w_D)^\\top$:\n", |
There was a problem hiding this comment.
I think introducing "t" is more confusing than helping - it is basically the same as y, from a rough semantic perspective, but a scalar rather than a function. I would not introduce it, instead I would all the function that you currently call "y" a function "f", and the nintroduce data "y" later, to whic hthe model is fit.
Further, I would use notation
| "\n", | ||
| "$$\n", | ||
| "\\begin{aligned}\n", | ||
| "p(t | \\mathbf{x}, \\mathbf{w}, \\beta) &= \\mathcal{N}(t | y(\\mathbf{x}, \\mathbf{w}), \\beta^{-1}) \\\\\n", |
There was a problem hiding this comment.
this notation for N is odd, I think incorrect?
What is it supposed to mean?
- Should "t" be a free variable instead?
- N usually stands for the distibution, not the pdf.
| " The second notebook delves into the concept of conjugate priors. By using conjugate distributions, we can derive posterior distributions analytically, simplifying Bayesian inference. This notebook also highlights how prior knowledge can influence the model and improve predictions in the presence of limited data.\n", | ||
| "\n", | ||
| "\n", | ||
| "3. **MCMC and Variational Inference** \n", |
There was a problem hiding this comment.
there are also 4. various strategies known as "approximate Bayes" here.
There was a problem hiding this comment.
Nice notebooks!
High-level, I think two points of improvement are most important:
- it is crucial that a reader gets some signposting about what they can expect from the notebooks, otherwise notebook 1 may disappoint expectations. A typical reader - seeing the
bayesianfolder - will want to see how they can useskpro. However, that is not what notebook 1 is about.- it is still interesting, but perhaps say at the start that this is more of an explainer intro with code.
- for code notebooks, the textbook style is discouraged, at least I would recommend to use more telegram and bullet point style. That makes the notebook less of a lengthy read.
On the mechanical side:
- in notebook 2, I would suggest to minimize extraneous content that is added only for it. For instance, can we not use one of the
sklearndatasets, instead of adding csvs in the folder? - we should also minimize "line-by-line" logic in the cells, 20 lines of data generation is not very useful from a didactic perspective. Can we outsource this somewhere in a data generation routine, in the main repo? Or just use a dummy dataset from
sklearnand split it up? - It is also important to split the vignettes more, e.g., have
skprovignettes separate from plotting cells. - for plotting distributions - at least univariate ones - you can use the
skproonboard plotting. If you think it is not sufficient, we should add more features to that.- multivariate is not yet supported, but that might be also a good small project?
- Regarding the final summary, with advantages and disadvantages: conjucates are also simple, compute efficient algorithms typically. A user may want to use evaluation to see how well they fare, against more memory or compute heavy variants...
fkiraly
added
module:regression
fkiraly
changed the title
Reference Issues/PRs
Provides example for Bayesian Conjugate Linear Regressor #500
What does this implement/fix? Explain your changes.
This contribution includes the first two notebooks in a planned series of four. These two notebooks cover:
BayesianConjugateLinearRegressorestimator.In addition, lightweight
utils.pyand small synthetic dataset are included.Does your contribution introduce a new dependency? If yes, which one?
None
What should a reviewer concentrate their feedback on?
Correctness of exposition
Did you add any tests for the change?
No
Any other comments?
No
PR checklist
No
For all contributions
How to: add yourself to the all-contributors file in the
skproroot directory (not theCONTRIBUTORS.md). Common badges:code- fixing a bug, or adding code logic.doc- writing or improving documentation or docstrings.bug- reporting or diagnosing a bug (get this pluscodeif you also fixed the bug in the PR).maintenance- CI, test framework, release.See here for full badge reference
For new estimators
docs/source/api_reference/taskname.rst, follow the pattern.Examplessection.python_dependenciestag and ensureddependency isolation, see the estimator dependencies guide.