Replication data for: How cross-validation can go wrong and what to do about it
| Item Type: | Dataset |
|---|---|
| Title: | Replication data for: How cross-validation can go wrong and what to do about it |
| Date: | 2018 |
| Creator: |
Neunhoeffer, Marcel  ORCID: 0000-0002-9137-5785 ; Sternberg, Sebastian ORCID: 0000-0003-4225-6402
|
| Divisions: | Außerfakultäre Einrichtungen > Graduate School of Economic and Social Sciences- CDSS (Social Sciences) School of Social Sciences > Politische Wissenschaft, Quantitative Sozialwissenschaftliche Methoden (Gschwend 2007-) |
| DDC Classification: |
320 Political science |
|---|---|
| Abstract: | The introduction of new “machine learning” methods and terminology to political science complicates the interpretation of results. Even more so, when one term – like cross-validation – can mean very different things. We find different meanings of cross-validation in applied political science work. In the context of predictive modeling, cross-validation can be used to obtain an estimate of true error or as a procedure for model tuning. Using a single cross-validation procedure to obtain an estimate of the true error and for model tuning at the same time leads to serious misreporting of performance measures. We demonstrate the severe consequences of this problem with a series of experiments. We also observe this problematic usage of cross-validation in applied research. We look at Muchlinski et al. (2016) on the prediction of civil war onsets to illustrate how the problematic cross-validation can affect applied work. Applying cross-validation correctly, we are unable to reproduce their findings. We encourage researchers in predictive modeling to be especially mindful when applying cross-validation. (2018) (English) |
| External Identifier for Data: | https://doi.org/10.7910/DVN/Y9KMJW |
| URL: | https://madata.bib.uni-mannheim.de/682/ |
|---|---|
| Access (Controlled): | Only Metadata |
| License (Controlled): | Creative Commons: CC0 | Universal 1.0 (recommended) |
| Related Publication(s) in MADOC: | Neunhoeffer, Marcel und Sternberg, Sebastian (2019), How cross-validation can go wrong and what to do about it |
Full text not available from this repository.
| Date Deposited: | 08 Apr 2026 16:30 |
|---|---|
| Last Modified: | 08 Apr 2026 16:30 |
You have found an error? Please let us know about your desired correction here: E-Mail
Actions (login required)
 |
View Item |
