Statistical Significance

Statistical significance is the likelihood that a relationship between two or more variables is caused by something other than chance. Statistical hypothesis testing is used to determine whether the result of a data set is statistically significant. This test provides a p-value, representing the probability that random chance could explain the result. In general, a p-value of 5% or lower is considered to be statistically significant.

Statistical significance is used to accept or reject the null hypothesis, which hypothesizes that there is no relationship between measured variables. A data set is statistically significant when the set is large enough to accurately represent the phenomenon or population sample being studied. A data set is typically deemed to be statistically significant if the probability of the phenomenon being random is less than 1/20, resulting in a p-value of 5%. When the test result exceeds the p-value, the null hypothesis is accepted. When the test result is less than the p-value, the null hypothesis is rejected.

Suppose Joe Sample works for a company that manufactures running shoes. For optimal production, he considers how many shoes should be made in each gender's size. Joe does not rely on anecdotal evidence that males have bigger sizes relative to females; he opts to use a statistical study that shows the correlation between gender and foot size to make accurate forecasts.

If the study's p-value was 2% (<5%), it would have a statistically significant result. The p-value indicates there is only a 2% chance that the connection between foot size and gender was the result of chance. He could then reasonably use the study's data to prepare his company's production plans.

On the other hand, if the p-value was 6% (>5%), it would not be reasonable to use the study as a basis for his production plans. So, if the study with the 2% p-value said that most men have shoe sizes between 8 and 12 and women have shoe sizes between 4 and 8, he could develop plans to produce most of the shoes in those sizes.

[Important: Statistical significance is often used for new pharmaceutical drug trials, to test vaccines, and in the study of pathology for effectiveness testing and to inform investors on how successful the company is at releasing new products.]

For example, Novo Nordisk, the pharmaceutical leader in diabetes medication, reported that there was a statistically significant reduction in type 1 diabetes when it tested its new insulin. The test consisted of 26 weeks of randomized therapy among diabetes patients. The result was a reduction in type 1 diabetes and a p-value of less than 5%, meaning that the reduction in diabetes was not due to random chance.

• Statistical significance is the likelihood that a relationship between two or more variables is caused by something other than chance.
• Statistical significance is used to accept or reject the null hypothesis, which hypothesizes that there is no relationship between measured variables.
• Statistical hypothesis testing is used to determine whether the result of a data set is statistically significant.