Item Details

Effects of task type on morphosyntactic complexity across proficiency : evidence from a large learner corpus of A1 to C2 writings

Issue: Vol 3 No. 2 (2019) Special Issue: Linguistic complexity and instruction in second language acquisition

Journal: Instructed Second Language Acquisition

Subject Areas:

DOI: 10.1558/isla.38248


This study investigates the effect of instructional design on (morpho)syntactic complexity in second language (L2) writing development. We operationalised instructional design in terms of task type and empirically based the investigation on a large subcorpus (669,876 writings by 119,960 learners from 128 tasks at all Common European Framework of Reference for Languages levels) of the EF-Cambridge Open Language Database (EFCAMDAT; Geertzen, Alexopoulou and Korhonen 2014).

First, the 128 task prompts were manually categorised for task type (e.g. argumentation, description). Next, developmental trajectories of syntactic complexity from A1 to C2 were established using a variety of global (e.g. mean length of clause) and specific (e.g. non-third person singular present tense verbs) measures extracted using natural language processing techniques. The effects of task type were analysed using the categorisation from the first step. Finally, tasks that showed atypical behaviour for a measure given their task type were explored qualitatively.

Our results partially confirm earlier experimental and corpus-based studies (e.g. subordination associated with argumentative tasks). Going beyond, our large-scale data-driven analysis made it possible to identify specific measures that were naturally prompted by instructional design (e.g. narrations eliciting wh-phrases). We discuss which measures typically align with certain task types and highlight how instructional design relates to L2 developmental trajectories over time.

Author: Marije Michel, Akira Murakami, Theodora Alexopoulou, Detmar Meurers

View Full Text

References :

Alexopoulou, T., Geertzen, J., Korhonen, A. and Meurers, D. (2015) Exploring big educational learner corpora for SLA research: perspectives on relative clauses. International Journal of Learner Corpus Research 1(1): 96–129.

Alexopoulou, T., Michel, M., Murakami, A. and Meurers, D. (2017) Task effects on linguistic complexity and accuracy: a large‐scale learner corpus analysis employing natural language processing techniques. Language Learning 67(s1): 180–208.

Baralt, M., Gilabert, R. and Robinson, P. (eds) (2014) Task Sequencing and Instructed Second Language Learning. London: Bloomsbury Publishing.

Benson, P. and Reinders, H. (eds) (2011) Beyond the Language Classroom. London: Palgrave Macmillan.

Biber, D., Gray, B and Staples, S. (2014) Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics 37(5): 639–68.

Brezina, V. and Flowerdew, L. (eds) (2017) Learner Corpus Research: New Perspectives and Applications. London: Bloomsbury Academic.

Bulté, B. and Housen, A. (2012) Defining and operationalising L2 complexity. In A. Housen, F. Kuiken and I. Vedder (eds) Dimensions of L2 Performance and Proficiency 21–46. Amsterdam: John Benjamins Publishing Company.

Bulté, B. and Housen, A. (2014) Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing 26: 42–65.

Bulté, B. and Housen, A. (2018) Syntactic complexity in L2 writing: individual pathways and emerging group trends. International Journal of Applied Linguistics 28(1): 147–64.

Byrnes, H., Maxim, H. H., & Norris, J. M. (eds) (2010) Realizing advanced foreign language writing development in collegiate education: curricular design, pedagogy, assessment [Special issue]. Modern Language Journal 94(s1).

Chen, X. and Meurers, D. (2016) CTAP: a web-based tool supporting automatic complexity analysis. Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity 113–19. Osaka, Japan.

Connor-Linton, J. and Polio, C. (2014) Comparing perspectives on L2 writing: multiple analyses of a common corpus. Journal of Second Language Writing 26: 1–9.

Crossley, S. A. and McNamara, D. S. (2014) Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing 26: 66–79.

Cumming, A. (1990) Expertise in evaluating second language compositions. Language Testing 7(1): 31–51.

Ferris, D. R. (1994) Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly 28: 414–20.

Foster, P. and Skehan, P. (1996) The influence of planning and task type on second language performance. Studies in Second Language Acquisition 18: 299–323.

Frear, M. W. and Bitchener, J. (2015) The effects of cognitive task complexity on writing complexity. Journal of Second Language Writing 20: 45–57.

Gablasova, D., Brezina, V. and McEnery, T. (2017a) Exploring learner language through corpora: comparing and interpreting corpus frequency information. Language Learning 67(s1): 130–54.

Gablasova, D., Brezina, V., McEnery, T. and Boyd, E. (2017b) Epistemic stance in spoken L2 English: the effect of task and speaker style. Applied Linguistics 38(5): 613–37.

Geertzen, J., Alexopoulou, T. and Korhonen, A. (2014) Automatic linguistic annotation of large scale L2 databases: the EF-Cambridge Open Language Database (EFCAMDAT). Proceedings of the 31st Second Language Research Forum (SLRF). Pittsburgh, PA: Cascadilla Press.

Graesser, A. C., McNamara, D. S., Louwerse, M. M. and Cai, Z. (2004) Coh-Metrix: analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers 36: 193–202.

Granger, S., Gilquin, G. and Meunier, F. (2015) The Cambridge Handbook of Learner Corpus Research. Cambridge: Cambridge University Press.

Hinkel, E. (2009) The effects of essay topics on modal verb uses in L1 and L2 academic writing. Journal of Pragmatics 41(4): 667–83.

Hommel, G. (1988) A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75(2): 383–6.

Kormos, J. (2011) Task complexity and linguistic and discourse features of narrative writing performance. Journal of Second Language Writing 20(2): 148–61.

Kuiken, F. and Vedder, I. (2008) Cognitive task complexity and written output in Italian and French as a foreign language. Journal of Second Language Writing, 17(1): 48–60.

Kyle, K. (2016) Measuring syntactic development in L2 writing: fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication. Doctoral dissertation, Georgia State University. Retrieved on 19 September 2019 from

Lee, C. and Polio, J. (2017) Written language learning. In S. Loewen and M. Sato (eds) The Routledge Handbook of Instructed Second Language Acquisition 299–318. New York: Routledge.

Loewen, S. (2015) Introduction to Instructed Second Language Acquisition. New York: Routledge.

Loewen, S. and Sato, M. (2017) The Routledge Handbook of Instructed Second Language Acquisition. New York: Routledge.

Loschky, L. and Bley-Vroman, R. (1993) Grammar and task-based methodology. In G. Crookes and S. Gass (eds) Tasks and Language Learning: Integrating Theory and Practice 123–67. Bristol: Multilingual Matters.

Lu, X. (2010) Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics 15(4): 474–96.

Lu, X. (2011) A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly 45(1): 36–62.

Manchón, R. M. (2011) Learning-to-Write and Writing-to-Learn in an Additional Language. Amsterdam: John Benjamins Publishing Company.

McCarthy, P. M. and Jarvis, S. (2010) MTLD, vocd-D, and HD-D: a validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods 42: 381–92.

Meurers, D. (2015) Learner corpora and natural language processing. In S. Granger, G. Gilquin and F. Meunier (eds) The Cambridge Handbook of Learner Corpus Research 537–66. Cambridge: Cambridge University Press.

Meurers, D. and Dickinson, M. (2017) Evidence and interpretation in language learning research: opportunities for collaboration with computational linguistics. Language Learning 67(2): 67–96.

Michel, M. (2017) Complexity, accuracy and fluency in L2 production. In S. Loewen and M. Sato (eds) Routledge Handbook of Instructed Second Language Acquisition 50–68. New York: Routledge.

Murakami, A. (2016) Modeling systematicity and individuality in nonlinear second language development: the case of English grammatical morphemes. Language Learning 66(4): 834–71.

Murakami, A. and Alexopoulou, T. (2016) Longitudinal L2 development of the English article in individual learners. In A. Papafragou, D. Grodner, D. Mirman and J. Trueswell (eds) Proceedings of the 38th Annual Meeting of the Cognitive Science Society 1050–5. Austin, TX: Cognitive Science Society.

Nassaji, H. (2016) Interactional feedback in second language teaching and learning: a synthesis and analysis of current research. Language Teaching Research 20(4): 535–62.

Norris, J. M. and Ortega, L. (2009) Towards an organic approach to investigating CAF in instructed SLA: the case of complexity. Applied Linguistics 30(4): 555–78.

Ott, N., Ziai, R. and Meurers, D. (2012) Creation and analysis of a reading comprehension exercise corpus: towards evaluating meaning in context. In T. Schmidt and K. Wörner (eds) Multilingual Corpora and Multilingual Corpus Analysis 47–69. Hamburg Studies in Multilingualism (HSM). Amsterdam, Netherlands: John Benjamins.

Pallotti, G. (2015) A simple view of linguistic complexity. Second Language Research 31(1): 117–34.

Pander Maat, H., Kraf, R., van den Bosch, A. P. J., Dekker, N., Gompel, M. V., Kleijn, S. D., ... and Sloot, K. (2014) T-Scan: a new tool for analyzing Dutch text. Computational Linguistics in the Netherlands (4):53–74. Retrieved on 19 September 2019 from

Polio, C. and Park, J.-H. (2016) Language development in second language writing. In R. M. Manchón and P. Matsuda (eds) Handbook of Second and Foreign Language Writing 287–306. New York: Routledge.

Polio, C. and Yoon, H.-J. (2018) The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics 28(1): 165–88.

Quixal, M. and Meurers, D. (2016) How can writing tasks be characterized in a way serving pedagogical goals and automatic analysis needs? CALICO Journal 33(1): 19–48.

Rebuschat, P., Meurers, D. and McEnery, T. (2017) Language learning research at the intersection of experimental, computational, and corpus-based approaches. Language Learning, 67(s1): 6–13.

Robinson, P. (2001) Task complexity, task difficulty, and task production: exploring interactions in a componential framework. Applied Linguistics 22(1): 27–57.

Sato M. and Loewen, S. (2019) Evidence-based Second Language Pedagogy: A Collection of Instructed Second Language Acquisition Studies. New York: Routledge.

Skehan, P. (2009) Modelling second language performance: integrating complexity, accuracy, fluency, and lexis. Applied Linguistics 30(4): 510–32.

Swain, M. and Lapkin, S. (2002) Talking it through: two French immersion learners’ response to reformulation. International Journal of Educational Research 37(3–4): 285–304.

Tracy-Ventura, N. and Myles, F. (2015) The importance of task variability in the design of learner corpora for SLA research. International Journal of Learner Corpus Research 1(1): 58–95.

Tracy-Ventura, N. and Paquot, M. (eds) (in preparation) The Routledge Handbook of Second Language Acquisition and Corpora. New York: Routledge.

Vajjala, S. and Meurers, D. (2012) On improving the accuracy of readability classification using insights from second language acquisition. In Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications (BEA) 163–73. Montréal, Canada: ACL. Retrieved on 19 September 2019 from

Vajjala, S. and Meurers, D. (2014) Readability assessment for text simplification: from analysing documents to identifying sentential simplifications. ITL – International Journal of Applied Linguistics 165(2): 194–222.

Vyatkina, N. (2012) The development of second language writing complexity in groups and individuals: a longitudinal learner corpus study. Modern Language Journal 96: 576–98.

Vyatkina, N., Hirschmann, H. and Golcher, F. (2015) Syntactic modification at early stages of L2 German writing development: a longitudinal learner corpus study. Journal of Second Language Writing 29: 28–50.

Way, D. P., Joiner, E. G. and Seaman, M. A. (2000) Writing in the secondary foreign language classroom: the effects of prompts and tasks on novice learners of French. Modern Language Journal 84(2): 171–84.

Weigle, S. C. (2002) Assessing Writing. Cambridge: Cambridge University Press.

Wieling, M. (2018) Analyzing dynamic phonetic data using generalized additive mixed modeling: a tutorial focusing on articulatory differences between L1 and L2 speakers of English. Journal of Phonetics 70: 86–116.

Wieling, M., Montemagni, S., Nerbonne, J. and Baayen, R. H. (2014) Lexical differences between Tuscan dialects and standard Italian: accounting for geographic and socio-demographic variation using generalized additive mixed modeling. Language 90: 669–92.

Williams, J. (2012) The potential role(s) of writing in second language development. Journal of Second Language Writing 21(4): 321–31.

Wolfe-Quintero, K., Inagaki, S. and Kim, H.-Y. (1998) Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity. Manoa, Hawaii: Second Language Teaching and Curriculum Center, University of Hawaii at Manoa.

Wood, S. N. (2017) Generalized Additive Models: An Introduction with R (2nd edn). Boca Raton, FL: Chapman and Hall/CRC.

Yang, W., Lu, X. and Weigle, S. C. (2015) Different topics, different discourse: relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing 28: 53–67.

Yoon, H.-J. and Polio, C. (2017) The linguistic development of students of English as a second language in two written genres. TESOL Quarterly 51(2): 275–301.