Replication Infrequency: Perspectives from Applied Linguistics Editors

Articles in Press

Document Type : Original Research Article

Authors

1 Faculty of Letters and Human Sciences, Shahid Beheshti University, Tehran, Iran

2 Allameh Tabataba'i University

Abstract

Replication studies play a crucial role in advancing scientific research, yet they are notably infrequent in applied linguistics (AL) journals. While editors serve as pivotal decision-makers in shaping academic publications, their perspectives on the scarcity of replication studies remain unexplored. This study aims to uncover insights from 27 editors-in-chief regarding the infrequency of replications in AL journals, shedding light on the challenges faced by replication research within the discipline. Thematic analysis revealed that editors identify a complex set of interrelated factors hindering the prevalence of replication studies, including predefined guidelines, limitations of academic journals, and an implicit bias towards impactful and original research. Further, editors recognize the critical role of replication in scientific research but emphasize that it is not a panacea for all research problems, urging its consideration as valuable tool alongside other rigorous research methods. These findings underscore the need for reform in the research culture of AL, advocating for increased replication studies and emphasizing transparency and rigor in research practices.

Keywords

References

Al-Hoorie, Ali, and Emma Marsden. 2024. “Open scholarship and transparency in applied linguistics research.” In Encyclopedia of Applied Linguistics. Wiley-Blackwell.
Braun, Virginia, and Victoria Clarke. 2006. “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3 (2): 77–101. https://doi.org/10.1191/1478088706qp063oa 
Chambers, Christopher D., and Liana Tzavella. 2022. “The Past, Present and Future of Registered Reports.” Nature Human Behaviour 6 (1): 29–42.
Dörnyei, Zoltán, and Dale T. Griffee. 2010. Research Methods in Applied Linguistics. Oxford: Oxford University Press.
Egbert, Joy. 2007. “Quality Analysis of Journals in TESOL and Applied Linguistics.” TESOL Quarterly 41 (1): 157–171. https://doi.org/10.2307/40264335.
Frias-Navarro, Dolores, Jorge Pascual-Llobell, María Pascual-Soler, Javier Perezgonzalez, and Jorge Berrios-Riquelme. 2020. “Replication Crisis or an Opportunity to Improve Scientific Production?” European Journal of Education 55: 618–631. https://doi.org/10.1111/ejed.12417.
Gabelica, Maja, Roko Bojčić, and Livia Puljak. 2022. “Many Researchers Were Not Compliant with Their Published Data Sharing Statement: Mixed-Methods Study.” Journal of Clinical Epidemiology 150: 33–41. https://doi.org/10.1016/j.jclinepi.2022.05.019.
Hensel, Paul G. 2019. “Supporting Replication Research in Management Journals: Qualitative Analysis of Editorials Published between 1970 and 2015.” European Management Journal 37 (1): 45–57. https://doi.org/10.1016/j.emj.2018.03.004.
Hubbard, Raymond, and J. Scott Armstrong. 1994. “Replications and Extensions in Marketing: Rarely Published but Quite Contrary.” International Journal of Research in Marketing 11: 233–248. https://doi.org/10.1016/0167-8116(94)90003-5.
Isbell, Daniel R., Dan Brown, Meng Chen, Deryle J. Derrick, Rania Ghanem, Mariana N. G. Arvizu, et al. 2022. “Misconduct and Questionable Research Practices: The Ethics of Quantitative Data Handling and Reporting in Applied Linguistics.” The Modern Language Journal 106 (1): 172–195. https://doi.org/10.1111/modl.12760 
Liu, Meng, and Emma Marsden. 2024. "The Open Turn: Rethinking Applied Linguistics Research through Open Scholarship." In *Open Science and Linguistics*, edited by Alejandro Curado Fuentes, María Mercedes Rico García, and L. Fielden Burns, 35–58. Applied Linguistics Press. https://doi.org/10.31219/osf.io/9kqvf 
Madden, Charles S., Robert W. Easley, and Melvin G. Dunn. 1995. “How Journal Editors View Replication Research.” Journal of Advertising 24 (4): 77–87.
Marefat, Fahimeh, Mohammad Hassanzadeh, and Farzaneh Hamidi. 2024. “Incorporating Replication in Higher Education: Supervisors’ Perspectives and Institutional Pressures.” Accountability in Research. https://doi.org/10.1080/08989621.2024.2412054. 
Marsden, Emma, and Kara Morgan-Short. 2023. “(Why) Are Open Research Practices the Future for the Study of Language Learning?” Language Learning. Advance online publication. https://doi.org/10.1111/lang.12487.
Marsden, Emma, Kara Morgan‐Short, Silvina Thompson, and Dalia Abugaber. 2018. “Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field.” Language Learning 68 (2): 321–391. https://doi.org/10.1111/lang.12286.
Marsden, Emma, Kara Morgan-Short, Pavel Trofimovich, and Nick C. Ellis. 2018. “Introducing Registered Reports at Language Learning: Promoting Transparency, Replication, and a Synthetic Ethic in the Language Sciences.” Language Learning 68 (2): 309–320. https://doi.org/10.1111/lang.12284. 
Martin, G. Neil, and Richard M. Clarke. 2017. “Are Psychology Journals Anti-Replication? A Snapshot of Editorial Practices.” Frontiers in Psychology 8: 523. https://doi.org/10.3389/fpsyg.2017.00523.
McManus, Kevin. 2022. “Are Replication Studies Infrequent Because of Negative Attitudes? Insights from a Survey of Attitudes and Practices in Second Language Research.” Studies in Second Language Acquisition 43 (2): 335–359. https://doi.org/10.1017/S0272263121000375.
Miyakawa, Takashi. 2020. “No Raw Data, No Science: Another Possible Source of the Reproducibility Crisis.” Molecular Brain 13 (1): 1–6. https://doi.org/10.1186/s13041-020-0552-2.
Neuliep, James W., and Richard Crandall. 1993. “Reviewer Bias Against Replication Research.” Journal of Social Behavior and Personality 8 (6): 21–29.
Neuliep, James W., and Richard Crandall. 1991. “Editorial Bias Against Replication Research.” In Replication Research in the Social Sciences, edited by James W. Neuliep, 51–69. Thousand Oaks, CA: Sage Publications.
Nowell, Lorelli S., Jill M. Norris, Deborah E. White, and Nancy J. Moules. 2017. “Thematic Analysis: Striving to Meet the Trustworthiness Criteria.” International Journal of Qualitative Methods 16 (1). https://doi.org/10.1177/1609406917733847.
Oppenheim, Abraham N. 2000. Questionnaire Design, Interviewing and Attitude Measurement. London: Bloomsbury Publishing.
Plonsky, Luke. 2023. “Sampling and Generalizability in Lx Research: A Second-Order Synthesis.” Languages 8 (1): 75. https://doi.org/10.3390/languages8010075.
Porte, Graeme. 2013. “Who Needs Replication?” CALICO Journal 30 (1): 10–15.
Porte, Graeme K., ed. 2012. Replication Research in Applied Linguistics. Cambridge: Cambridge University Press.
Porte, Graeme K., and Kevin McManus. 2019. Doing Replication Research in Applied Linguistics. London: Routledge.
Stodden, Victoria, Jennifer Seiler, and Zhaokun Ma. 2018. “An Empirical Analysis of Journal Policy Effectiveness for Computational Reproducibility.” Proceedings of the National Academy of Sciences 115 (11): 2584–2589. https://doi.org/10.1073/pnas.1708290115.
Tipu, Syed Awais Ahmad, and James Christopher Ryan. 2022. "Are Business and Management Journals Anti-Replication? An Analysis of Editorial Policies." Management Research Review 45 (1): 101–117.https://doi.org/10.1108/MRR-01-2021-0050.
Weber, Melanie, and Christa M. Campbell. 2004. “In Other Professional Journals.” Modern Language Journal 88 (1): 130–139. https://doi.org/10.1111/j.0026-7902.2004.00219.x.
Yeung, Andy W. 2017. “Do Neuroscience Journals Accept Replications? A Survey of Literature.” Frontiers in Human Neuroscience 11: 468. https://doi.org/10.3389/fnhum.2017.00468.
Zimmerman, Marc A., and Gregory J. Zeitz. 2002. “Beyond Survival: Achieving New Venture Growth by Building Legitimacy.” Academy of Management Review 27 (3): 414–431.
Critical Language and Literary studies

Articles in Press, Accepted Manuscript
Available Online from 26 May 2025

Files

Share

How to cite

Statistics