ECML 2016 tutorial on Bayesian vs. Frequentist tests for comparing algorithms

Tutorial went very well. It was a nice experience and we received very positive feedback. If you are interested in the content please visit this page.

Clinton vs. Trump 23th Sptember 2016

 

I have run again the Python code that computes the worst-case (red) and best-Case (blue) posterior distribution for Clinton winning the general USA election. using fresh (September) poll-data.    At the moment there is a quite large uncertainty but is still in favour of Clinton: the  probability of winning is between 0.78 and 0.95.  If you are interested in the methodology I have used to compute these distributions please see the past posts. If you want to try yourself, Python code and data-poll from fivethirtyeight.com are available in my github (just click on the links).

 

19 September Tutorial at ECML

Working on the slides for our Tutorial at ECML 2016 (Riva del Garda)  


G. Corani, A. Benavoli, J. Demsar.  Comparing competing algorithms: Bayesian versus frequentist hypothesis testing

Schedule

Time	Duration	Content	Details
09:00	15min	Introduction	Motivations and Goals
09:15	60min	Null hypothesis significance tests in machine learning	NHST testing (methods and drawbacks)
10:15	25min	Introduction to Bayesian tests	Bayesian model comparison versus Bayesian estimation
10:40	20min	Break	Is the coffee in Riva del Garda better than the coffee in Porto?
11:00	35min	Bayesian hypothesis testing for comparing classifiers	Single and hierarchical Bayesian models
11:35	55min	Non-parametric Bayesian tests and presentation of the results of Bayesian analysis	Dirichlet process and how to perform nonparametric Bayesian tests
12:30	10min	Summarizing!	Summary and conclusions