Full, Keyword, or Glossed? Exploring How Captioning Modes Influence EFL Learners’ Listening Comprehension and Caption Reading Patterns: An Eye-Tracking Study

مقالات آماده انتشار

نوع مقاله : مقاله علمی پژوهشی

نویسندگان

1 دانشگاه شهید بهشتی

2 دانشگاه فریدریش شیلر ینا، وایمار، آلمان

3 دانشگاه شهید بهشتی تهران ایران

چکیده

This study investigated the effects of different captioning modes on intermediate English as a Foreign Language (EFL) learners’ listening comprehension and examined how caption reading patterns influence comprehension of audio-visual materials. Using a two-phase sequential quasi-experimental design, 76 Iranian intermediate EFL learners were assigned to three groups receiving full, keyword, or glossed captioned videos during a six-session instructional program. Listening comprehension was measured through IELTS-based tests, and a subsequent eye-tracking phase with 36 participants explored learners’ visual attention and caption reading behaviors. One-way ANOVA results indicated a statistically significant effect of captioning mode on listening comprehension, with full captioning leading to significantly higher performance than keyword captioning, while no significant differences were found between full and glossed captioning or between keyword and glossed captioning. Eye-tracking data revealed differences in caption reading patterns between high- and low-performing learners: high-performing learners demonstrated more selective attention, focusing primarily on verbs and spending less time on captions overall, whereas low-performing learners exhibited longer and more distributed attention across caption elements. The findings suggest that captioning mode and learners’ attention allocation play important roles in listening comprehension and that efficient caption use is associated with better learning outcomes. The study provides pedagogical implications for the use of captioned audio-visual materials and contributes to understanding cognitive processing in multimedia-assisted language learning.

کلیدواژه‌ها

عنوان مقاله [English]

Full, Keyword, or Glossed? Exploring How Captioning Modes Influence EFL Learners’ Listening Comprehension and Caption Reading Patterns: An Eye-Tracking Study

نویسندگان [English]

  • Musa Nushi 1
  • Kimia Askarian 2
  • Mehran Hosseinkhani 3
1 Shahid Beheshti University
2 Friedrich-Schiller-Universität Jena, Weimar, Germany
3 Shahid Beheshti university, Tehran, Iran
چکیده [English]

This study investigated the effects of different captioning modes on intermediate English as a Foreign Language (EFL) learners’ listening comprehension and examined how caption reading patterns influence comprehension of audio-visual materials. Using a two-phase sequential quasi-experimental design, 76 Iranian intermediate EFL learners were assigned to three groups receiving full, keyword, or glossed captioned videos during a six-session instructional program. Listening comprehension was measured through IELTS-based tests, and a subsequent eye-tracking phase with 36 participants explored learners’ visual attention and caption reading behaviors. One-way ANOVA results indicated a statistically significant effect of captioning mode on listening comprehension, with full captioning leading to significantly higher performance than keyword captioning, while no significant differences were found between full and glossed captioning or between keyword and glossed captioning. Eye-tracking data revealed differences in caption reading patterns between high- and low-performing learners: high-performing learners demonstrated more selective attention, focusing primarily on verbs and spending less time on captions overall, whereas low-performing learners exhibited longer and more distributed attention across caption elements. The findings suggest that captioning mode and learners’ attention allocation play important roles in listening comprehension and that efficient caption use is associated with better learning outcomes. The study provides pedagogical implications for the use of captioned audio-visual materials and contributes to understanding cognitive processing in multimedia-assisted language learning.

کلیدواژه‌ها [English]

  • audio-visual materials
  • captioning mode
  • eye-tracking
  • multimedia-assisted language learning
  • quasi-experimental design
  • visual attention

مراجع

Alabsi, Thuraya. 2020. “Effects of Adding Subtitles to Video via Apps on Developing EFL Students’ Listening Comprehension.” Theory and Practice in Language Studies 10 (10): 1191–1199. https://doi.org/10.17507/tpls.1010.02
Alamri, Wafa. 2025. “Evaluating the Benefits and Challenges of Using Authentic Materials in EFL Context for Listening Purpose.” Frontiers in Education 10: 1611308. https://doi.org/10.3389/feduc.2025.1611308
Aldera, Abdullah S., and Mohammed Ali Mohsen. 2013. “Annotations in Captioned Animation: Effects on Vocabulary Learning and Listening Skills.” Computers & Education 68: 60–75. https://doi.org/10.1016/j.compedu.2013.04.018
Alotaibi, Hind M., Hassan Saleh Mahdi, and Deema Alwathnani. 2023. "Effectiveness of Subtitles in L2 Classrooms: A Meta-Analysis Study" Education Sciences 13, no. 3: 274. https://doi.org/10.3390/educsci13030274
Aryadoust, Vahid, and Bee Hoon Ang. 2021. “Exploring the Frontiers of Eye Tracking Research in Language Studies: A Novel Co-Citation Scientometric Review.” Computer Assisted Language Learning 34 (7): 898–933. https://doi.org/10.1080/09588221.2019.1647251
Batty, Aaron Olaf. 2021. “An Eye-Tracking Study of Attention to Visual Cues in L2 Listening Tests.” Language Testing 38 (4): 511–535. https://doi.org/10.1177/0265532220951504
Conklin, Kathy, Ana Pellicer-Sánchez, and Gareth Carrol. 2019. Eye-Tracking: A Guide for Applied Linguistics Research. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108233279
Conklin, Kathy, Sara Alotaibi, Ana Pellicer-Sánchez, and Laura Vilkaitė-Lozdienė. 2020. “What Eye-Tracking Tells Us about Reading-Only and Reading-While-Listening in a First and Second Language.” Second Language Research 36 (3): 257–276. https://doi.org/10.1177/0267658320921496
Coyne, Joseph T., Noelle Brown, Cyrus K. Foroughi, and Ciara M. Sibley. 2019. “Improving Pupil Diameter Measurement Accuracy in a Remote Eye Tracking System.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63 (1): 49–53. https://doi.org/10.1177/1071181319631176
Danan, Martine. 2016. “Enhancing Listening with Captions and Transcripts.” Applied Language Learning 26: 1–24.
d’Ydewalle, Géry, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults While Reading Television Subtitles.” European Psychologist 12 (3): 196–205. https://doi.org/10.1027/1016-9040.12.3.196
Dolgunsöz, Emrah. 2015. “Measuring Attention in Second Language Reading Using Eye-Tracking: The Case of the Noticing Hypothesis.” Journal of Eye Movement Research 8 (5). https://doi.org/10.16910/jemr.8.5.4
Dong, Jiangqiao, Yajing Zhou, and Guiru Liu. 2015. “The Effect of Caption Modes on EFL Students’ Video Comprehension.” Journal of Language Teaching and Research 6 (2): 397–404. http://dx.doi.org/10.17507/jltr.0602.21
Duchowski, Andrew T. 2002. “A Breadth-First Survey of Eye-Tracking Applications.” Behavior Research Methods, Instruments, & Computers 34: 455–470. https://doi.org/10.3758/BF03195475
Field, John. 2019. “Second Language Listening: Current Ideas, Current Issues.” In The Cambridge Handbook of Language Learning, edited by John W. Schwieter and Alessandro Benati, 283–319. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108333603.013
Garza, Thomas J. 1991. “Evaluating the Use of Captioned Video Materials in Advanced Foreign Language Learning.” Foreign Language Annals 24 (3): 239–258. https://doi.org/10.1111/j.1944-9720.1991.tb00469.x
Gass, Susan, Paula Winke, Daniel R. Isbell, and Jieun Ahn. 2019. “How Captions Help People Learn Languages: A Working-Memory, Eye-Tracking Study.” Language Learning & Technology 23 (2): 84–104. https://doi.org/10.10125/44684
Gruba, Paul. 2006. “Playing the Videotext: A Media Literacy Perspective on Video-Mediated L2 Listening.” Language Learning & Technology 10 (2): 77–92. https://doi.org/10.64152/10125/44062
Guillory, Helen Gant. 1998. “The Effects of Keyword Captions to Authentic French Video in Foreign Language Instruction.” CALICO Journal 15: 89–108.
Inhoff, Albrecht W., Matthew S. Starr, Matthew Solomon, and Lars Placke. 2008. “Eye Movements during the Reading of Compound Words and the Influence of Lexeme Meaning.” Memory & Cognition 36 (3): 675–687. https://doi.org/10.3758/mc.36.3.675
Just, Marcel, and Patricia A. Carpenter. 1980. A Theory of Reading: From Eye Fixations to Comprehension. Pittsburgh: Carnegie Mellon University. https://doi.org/10.1184/R1/6613262.v1
Joseph, Holly S. S. L., Simon P. Liversedge, Hazel I. Blythe, Sarah J. White, Susan E. Gathercole, and Keith Rayner. 2008. “Children’s and Adults’ Processing of Anomaly and Implausibility during Reading: Evidence from Eye Movements.” The Quarterly Journal of Experimental Psychology 61 (5): 708–723. https://doi.org/10.1080/17470210701400657
Li, Yan. 2025. "Listen or Read? The Impact of Proficiency and Visual Complexity on Learners’ Reliance on Captions" Behavioral Sciences 15, no. 4: 542. https://doi.org/10.3390/bs15040542
Liu, Tingting, and Vahid Aryadoust. “An Eye-Tracking Study of the Impact of Item Presentation and Item Format on Cognitive Processing in L2 Listening Assessment.” Studies in Second Language Acquisition, 2026, 1–28. https://doi.org/10.1017/S0272263126101648.
Loewen, Shawn, and Luke Plonsky. 2017. An A–Z of Applied Linguistics Research Methods. London: Bloomsbury Academic. https://doi.org/10.1007/978-1-137-40322-3_1
Mahalingappa, Laura, Jiaxuan Zong, and Nihat Polat. 2024. “The Impact of Captioning and Playback Speed on Listening Comprehension of Multilingual English Learners at Varying Proficiency Levels.” System 120: 103192. https://doi.org/10.1016/j.system.2023.103192
Maran, Thomas, Marco Furtner, Simon Liegl, Theo Ravet-Brown, Lucas Haraped, and Pierre Sachse. 2021. “Visual Attention in Real-World Conversation: Gaze Patterns Are Modulated by Communication and Group Size.” Applied Psychology 70: 1602–1627. https://doi.org/10.1111/apps.12291
Mashat, Arwa. 2024. “Eye Physiology, Movement, and the Role of Eye Tracking in Instructional Design.” In Navigating the World of Multimedia: Innovation and Applications. IntechOpen. https://doi.org/10.5772/intechopen.1008301
Mayer, Richard E. 2020. Multimedia Learning (3rd ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/9781316941355
Montero Perez, Maribel, Elke Peters, and Piet Desmet. 2015. “Enhancing Vocabulary Learning through Captioned Video: An Eye-Tracking Study.” The Modern Language Journal 99: 308–328. https://doi.org/10.1111/modl.12215
Montero Perez, Maribel, Elke Peters, and Piet Desmet. 2014. “Is Less More? Effectiveness and Perceived Usefulness of Keyword and Full Captioned Video for L2 Comprehension.” ReCALL 26 (1): 21–43. https://doi.org/10.1017/S0958344013000256
Muñoz, Carmen. 2017. “The Role of Age and Proficiency in Subtitle Reading: An Eye-Tracking Study.” System 67: 77–86. https://doi.org/10.1016/j.system.2017.04.015
Nushi, Musa, and Fereshte Orouji. 2020. “Investigating EFL Teachers’ Views on Listening Difficulties among Their Learners: The Case of Iranian Context.” SAGE Open 10 (2). https://doi.org/10.1177/2158244020917393
Park, Myongsu. 2004. “The Effects of Partial Captions on Korean EFL Learners’ Listening Comprehension.” PhD diss., Dissertation Abstracts International, A: The Humanities and Social Sciences 65 (4): 1287-A. (UMI Order No. DA3128868).
Rayner, Keith. 1998. “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124 (3): 372–422. https://doi.org/10.1037/0033-2909.124.3.372
Rost, Michael. 2024. Teaching and Researching Listening (4th ed.). New York: Routledge. https://doi.org/10.4324/9781003390794
Sweller, John. 1988. “Cognitive Load during Problem Solving: Effects on Learning.” Cognitive Science 12 (2): 257–285. https://doi.org/10.1207/s15516709cog1202_4
Sweller, John. 1999. Instructional design in technical areas. Camberwell, Australia: ACER Press.
Taylor, Gregory. 2005. “Perceived Processing Strategies of Students Watching Captioned Video.” Foreign Language Annals 38 (3): 422–427. https://doi.org/10.1111/j.1944-9720.2005.tb02228.x
Teng, Mark Feng. 2024. “Captioned Viewing for Language Learning: A Cognitive and Affective Model.” In Innovations in Technologies for Language Teaching and Learning, edited by Hung Phu Bui and Ehsan Namaziandost, Studies in Computational Intelligence 1159. Cham: Springer. https://doi.org/10.1007/978-3-031-63447-5_1
Teng, Mark Feng. 2022. “Vocabulary Learning through Videos: Captions, Advance-Organizer Strategy, and Their Combination.” Computer Assisted Language Learning 35 (3): 518–550. https://doi.org/10.1080/09588221.2020.1720253
Vanderplank, Robert. 2016. “‘Effects of’ and ‘Effects with’ Captions: How Exactly Does Watching a TV Programme with Same-Language Subtitles Make a Difference to Language Learners?” Language Teaching 49 (2): 235–250. https://doi.org/10.1017/S0261444813000207
Weingärtner, Henrike, Maximiliane Windl, Lewis L. Chuang, and Fiona Draxler. 2024. “Useful but Distracting: Viewer Experience with Keyword Highlights and Time-Synchronization in Captions for Language Learning.” In Proceedings of MUM 2024: The 23rd International Conference on Mobile and Ubiquitous Multimedia, December 1–4, Stockholm, Sweden, 235–248. https://doi.org/10.1145/3701571.3701574
Winke, Paula M. 2013. “The Effects of Input Enhancement on Grammar Learning and Comprehension: A Modified Replication of Lee (2007) with Eye-Movement Data.” Studies in Second Language Acquisition 35 (2): 323–352. https://doi.org/10.1017/S0272263112000903
Winke, Paula, Susan Gass, and Tetyana Sydorenko. 2013. “Factors Influencing the Use of Captions by Foreign Language Learners: An Eye-Tracking Study.” The Modern Language Journal 97 (1): 254–275. https://doi.org/10.1111/j.1540-4781.2013.01432.x
Winke, Paula, Susan Gass, and Tetyana Sydorenko. 2010. “The Effects of Captioning Videos Used for Foreign Language Listening Activities.” Language Learning & Technology 14: 66–87.
Wu, Huizhen, Ping Yu, Shenshen Yang, and Xuanyuan Chen. 2022. “Video Captioning Effects on EFL Listening Comprehension and Vocabulary Learning: Help or Hurdle?” International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT) 12 (2): 1–16. https://doi.org/10.4018/IJCALLT.291534
Yeldham, Michael. 2018. “Viewing L2 Captioned Videos: What’s in It for the Listener?” Computer Assisted Language Learning 31 (4): 367–89. https://doi.org/10.1080/09588221.2017.1406956
 
نقد زبان و ادبیات خارجی

مقالات آماده انتشار، پذیرفته شده
انتشار آنلاین از تاریخ 15 تیر 1405

فایل ها

هم رسانی

ارجاع به این مقاله

آمار