Zad Rafi Chow
NYU Langone Medical Center
Co-Authors: Andrew Vigotsky, Sander Greenland
How Surprising Are Your Data? Cognitive and Graphical Tools to Aid Applied Researchers
Researchers often misinterpret inferential statistics as evidence for the absence or discovery of effects. Much of the discussion around these issues has been on replacing tools such as P-values with confidence intervals, Bayes factors, or changing thresholds for significance.
We believe these issues are cognitive rather than statistical. Thus, we propose some cognitive and graphical tools to aid researchers in interpreting their data from an informational perspective, rather than a probabilistic one.
Our proposals are simple transformations of the P-value and its function. They are obtained by taking the -log_2 of the P-value, yielding something known as the surprisal (s) value.
This value provides bits of information against the test hypothesis and can be interpreted within the context of coin flips, reducing confusion for a posterior probability. Transforming a P-value function into a surprisal function may also prevent confusion of absence of evidence for evidence of absence.