Workshop: What will inspire us to switch from unhelpful metrics to enlightening metrics?

ABSTRACT:

Our profession has habits of using particular metrics even when the metrics do not address our research questions. Professors pass these habits to students who perpetuate the dysfunctional culture. These routinely-abused popular metrics include: kappa, area under the ROC curve, F1, root mean squared error, Nash-Sutcliffe Index, overall agreement, and p-value. Some authors seem to think that they must claim that their results have high agreement, but the opportunity to advance the profession is to acknowledge error. This presentation gives examples of dysfunctional practices and recommends straightforward fixes. The mathematical solutions are clear, but the adoption of the solutions is slow because of sociological problems among scientists in the profession.

BIOGRAPHY:

Robert Gilmore Pontius Jr is a Professor in the Graduate School of Geography at Clark University in the United States. He specializes in Land Change Science, Geographical Information Science, and Statistics. The United States National Science Foundation has funded his research through the Long Term Ecological Research network. His newest research collaboration with MapBiomas is creating methods and software to analyze land change within a time series of land cover maps. Gil derives mathematical methods, some of which are in Clark University’s TerrSet software, which has more than one hundred thousand users worldwide. He has published more than ninety journal articles. His most popular article is entitled “Death to Kappa”, which is controversial because scientists routinely use the Kappa index of agreement even when Kappa is misleading. He is a co-author of the book “Advancing Land Change Modeling” and has recently published a solo-authored book entitled “Metrics That Make a Difference: How to Analyze Change and Error”. Pontius will describe several of the book’s ideas in a workshop at this conference.