Item Details

Analyzing Processing Time Data in Applied Linguistics and Second Language Research: A Multivariate Mixed-effects Approach

Issue: Vol 5 No. 1-2 (2018)

Journal: Journal of Research Design and Statistics in Linguistics and Communication Science

Subject Areas: Linguistics

DOI: 10.1558/jrds.39117

Abstract:

In this paper, I introduce the use of an extension of the popular linear mixed-effects models: multivariate mixed models which can handle multiple outcome variables. This technique is especially useful for applied linguists and second language researchers who use processing time data (e.g., accuracy and reaction times) obtained from timed decision tasks, text-based eye tracking, and self-paced reading. It can address the long-standing issue of multiple comparisons as a result of having multiple outcomes (e.g., first fixation durations and total reading times in text-based eye tracking, and time spent in different regions in self-paced reading). This technique also provides exciting opportunities for researchers to ask new questions that could not be addressed in a straightforward manner with traditional statistics. With this technique, researchers are able to investigate differential effects of a predictor on different outcomes. Through a demonstration in R using published, open eye-tracking data, I contextualize my discussion of the technique, offering also practical, step-by-step, and annotated guidelines for interested researchers.

Author: Bronson Hui

View Full Text

References :

Baayen, R. H. (2008). Analyzing Linguistic Data: A Practical Introduction to Statistics using R. Cambridge: Cambridge University Press.
https://doi.org/10.1017/CBO9780511801686

 

Baayen, R. H., Davidson, D. J., and Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59 (4): 390-412.
https://doi.org/10.1016/j.jml.2007.12.005

 

Baldwin, S. A., Imel, Z. E., Braithwaite, S. R., and Atkins, D. C. (2014). Analyzing multiple outcomes in clinical research using multivariate multilevel models. Journal of Consulting and Clinical Psychology 82 (5), 920-930.
https://doi.org/10.1037/a0035628

 

Barr, D. J., Levy, R., Scheepers, C., and Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language 68 (3): 255-278.
https://doi.org/10.1016/j.jml.2012.11.001

 

Cunnings, I. (2012). An overview of mixed-effects statistical models for second language researchers. Second Language Research Analyzing multiple outcomes in clinical research using multivariate multilevel models 28 (3): 369-382.
https://doi.org/10.1177/0267658312443651

 

Cunnings, I. and Finlayson, I. (2015). Mixed effects modeling and longitudinal data analysis. In L. Plonsky (Ed.), Advancing Quantitative Methods in Second Language Research, 159-181. New York: Routledge.
https://doi.org/10.4324/9781315870908-8

 

Draheim, C., Hicks, K. L., and Engle, R. W. (2016). Combining reaction time and accuracy: The relationship between working memory capacity and task switching as a case example. Perspectives on Psychological Science 11: 133-155.
https://doi.org/10.1177/1745691615596990

 

Draheim, C., Mashburn, C. A., Martin, J. D., and Engle, R. W. (2019). Reaction time in differential and developmental research: A review and commentary on the problems and alternatives. Psychological Bulletin 145 (5): 508-535.
https://doi.org/10.1037/bul0000192

 

Elgort, I., Brysbaert, M., Stevens, M., and Van Assche, E. (2018). Contextual word learning during reading in a second language: An eye-movement study. Studies in Second Language Acquisition 40 (2): 341-366.
https://doi.org/10.1017/S0272263117000109

 

Godfroid, A. (2019). Eye Tracking in Second Language Acquisition and Bilingualism: A Research Synthesis and Methodological Guide. New York: Routledge.

 

Godfroid, A. and Winke, P. (2015). Investigating implicit and explicit processing using
https://doi.org/10.1075/sibil.48.14god

 

L2 learners' eye-movement data. In P. Rebuschat (Ed.) Implicit and Explicit

 

Learning of Languages, 325-348. Amsterdam: John Benjamins.

 

Gries, S. Th. (2015). The most under-used statistical method in corpus linguistics: Multi-level (and mixed-effects) models. Corpora 10 (1): 95-125.
https://doi.org/10.3366/cor.2015.0068

 

Hadfield, J. D. (2010). MCMC methods for multi-response generalized linear mixed models: The MCMCglmm R package. Journal of Statistical Software 33 (2): 1-22.
https://doi.org/10.18637/jss.v033.i02

 

Hamaker, E. L. and Klugkist, I. (2011). Bayesian estimation of multilevel models. In J. J. Hox and J. K. Roberts (Eds), Handbook of Advanced Multilevel Analysis, 137-161. New York: Routledge.

 

Hox, J. J., Moerbeek, M., and Van de Schoot, R. (2018). Multilevel Analysis: Techniques and Applications. New York: Routledge.
https://doi.org/10.4324/9781315650982

 

Indrarathne, B., Ratajczak, M., and Kormos, J. (2018). Modelling changes in the cognitive processing of grammar in implicit and explicit learning conditions: Insights from an eye‐tracking study. Language Learning 68 (3): 669-708.
https://doi.org/10.1111/lang.12290

 

Jeon, E. H. (2015). Multiple regression. In L. Plonsky (Ed.), Advancing Quantitative Methods in Second Language Research, 131-158. New York: Routledge.
https://doi.org/10.4324/9781315870908-7

 

Jiang, N. (2013). Conducting Reaction Time Research in Second Language Studies. New York: Routledge.
https://doi.org/10.4324/9780203146255

 

Keating, G. and Jegerski, J. (2015). Experimental designs in sentence processing research. Studies in Second Language Acquisition 37: 1-32.
https://doi.org/10.1017/S0272263114000187

 

Lagrou, E., Hartsuiker, R. J., and Duyck, W. (2011). Knowledge of a second language influences auditory word recognition in the native language. Journal of Experimental Psychology. Learning, Memory, and Cognition 37 (4): 952-965.
https://doi.org/10.1037/a0023217

 

Larson-Hall, J. and Herrington, R. (2010). Improving data analysis in second language acquisition by utilizing modern developments in applied statistics. Applied Linguistics 31 (3): 368-390.
https://doi.org/10.1093/applin/amp038

 

Linck, J. A. and Cunnings, I. (2015). The utility and application of mixed-effects models in second language research. Language Learning 65 (S1), 185-207.
https://doi.org/10.1111/lang.12117

 

Marsden, E., Morgan-Short, K., Thompson, S., and Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning 68: 321-391.
https://doi.org/10.1111/lang.12286

 

Marsden, E., Thompson, S., and Plonsky, L. (2018). A methodological synthesis of self-paced reading in second language research. Applied Psycholinguistics 39 (5): 861-904.
https://doi.org/10.1017/S0142716418000036

 

Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., and Bates, D. (2017). Balancing Type I error and power in linear mixed models. Journal of Memory and Language 94: 305-315.
https://doi.org/10.1016/j.jml.2017.01.001

 

Norouzian, R., de Miranda, M., and Plonsky, L. (2018). The Bayesian revolution in second language research: An applied approach. Language Learning 68 (4): 1032-1075.
https://doi.org/10.1111/lang.12310

 

Plonsky, L. (2014). Study quality in quantitative L2 research (1990-2010): A methodological synthesis and call for reform. The Modern Language Journal 98 (1): 450-470.
https://doi.org/10.1111/j.1540-4781.2014.12058.x

 

Plonsky, L. and Oswald, F. L. (2017). Multiple regression as a flexible alternative to ANOVA in L2 research. Studies in Second Language Acquisition 39 (3): 579-592.
https://doi.org/10.1017/S0272263116000231

 

Polio, C. and Gass, S. M. (1997). Replication and reporting: A commentary. Studies in Second Language Acquisition 19: 499-508.
https://doi.org/10.1017/S027226319700404X

 

Porte, G. (2012). Replication Research in Applied Linguistics. Cambridge: Cambridge University Press.

 

Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed). London: Sage.

 

Rayner, K. (2009). Eye movements in reading: Models and data. Journal of Eye Movement Research 2: 1-10.

 

Rayner, K., Reichle, E. D., and Pollatsek, A. (2005). Eye movement control in reading and the E-Z Reader model. In G. Underwood (Ed.), Cognitive Processes in Eye Guidance, 131-162. Oxford: Oxford University Press.
https://doi.org/10.1093/acprof:oso/9780198566816.003.0006

 

Regev, S. and Meiran, N. (2014). Post-error slowing is influenced by cognitive control demand. Acta Psychologica 152: 10-18.
https://doi.org/10.1016/j.actpsy.2014.07.006

 

Ross, S. J. and Mackey, B. (2015). Bayesian approaches to imputation, hypothesis testing, and parameter estimation. Language Learning 65 (S1), 208-227.
https://doi.org/10.1111/lang.12118

 

Tabachnick, B. G. and Fidell, L. S. (2013). Using Multivariate Statistics (6th ed.) Boston, MA: Pearson.

 

Van de Schoot, R., Winter, S., Ryan, O., Zondervan-Zwijnenburg, M., and Depaoli, S. (2017). A systematic review of Bayesian papers in psychology: The last 25 years. Psychological Methods 22: 217-239.
https://doi.org/10.1037/met0000100

 

Von der Malsburg, T. and Angele, B. (2017). False positives and other statistical errors in standard analyses of eye movements in reading. Journal of Memory and Language 94: 119-133.
https://doi.org/10.1016/j.jml.2016.10.003

 

Whitford, V. and Titone, D. (2019). Lexical entrenchment and cross-language activation: Two sides of the same coin for bilingual reading across the adult lifespan. Bilingualism: Language and Cognition 22 (1): 58-77.
https://doi.org/10.1017/S1366728917000554

 

Winke, P. M., Godfroid, A., and Gass, S. M. (2013). Introduction to the special issue: Eye-movement recordings in second language research. Studies in Second Language Acquisition 35 (2): 205-212.
https://doi.org/10.1017/S027226311200085X