Archive | Statistical Methods

How to Use Statistical Significance Tests to Interpret Machine Learning Results

How to Use Statistical Significance Tests to Interpret Machine Learning Results

It is good practice to gather a population of results when comparing two different machine learning algorithms or when comparing the same algorithm with different configurations. Repeating each experimental run 30 or more times gives you a population of results from which you can calculate the mean expected performance, given the stochastic nature of most […]

Continue Reading 30
Zoomed Line Plot of Mean Result with Standard Error Bars and Population Mean

Estimate the Number of Experiment Repeats for Stochastic Machine Learning Algorithms

A problem with many stochastic machine learning algorithms is that different runs of the same algorithm on the same data return different results. This means that when performing experiments to configure a stochastic algorithm or compare algorithms, you must collect multiple results and use the average performance to summarize the skill of the model. This […]

Continue Reading 23