Item Details

Using Grammar Checkers in an ESL Context: An Investigation of Automatic Corrective Feedback

Issue: Vol 37 No. 2 (2020)

Subject Areas:

Abstract:

Our study examines written corrective feedback generated by two online grammar checkers (GCs), Grammarly and Virtual Writing Tutor, and by the grammar checking function of Microsoft Word. We tested the technology on a wide range of grammatical error types from two sources: a set of authentic ESL compositions and a series of simple sentences we generated ourselves. The GCs were evaluated in terms of (1) coverage (number of errors flagged), (2) appropriacy of proposed replacement forms, and (3) rates of “false alarms” (forms mistakenly flagged as incorrect). Although Grammarly and Virtual Writing Tutor outperformed Microsoft Word, neither of the online GCs had high rates of overall coverage (<50%). Consequently, they cannot be relied on to supply comprehensive feedback on student compositions. The finding of higher identification rates for errors from simple rather than authentic sentences reinforces this conclusion. Nonetheless, since few inaccurate replacement forms and false alarms were observed, only rarely is the feedback actively misleading. In addition, the GCs were better at handling some error types than others. Ultimately, we suggest that teachers use GCs with specially designed classroom activities that target selected error types before learners apply the technology to their own writing.

Author: Paul John, Nina Woll

View Full Text

References :

Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17, 102–118. https://doi.org/10.1016/j.jslw.2007.11.004

Bitchener, J., & Knoch, U. (2010). Raising the linguistic accuracy level of advanced L2 writers with written corrective feedback. Journal of Second Language Writing, 19, 207–217. https://doi.org/10.1016/j.jslw.2010.10.002

Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing. Journal of Second Language Writing, 12, 267–296. https://doi.org/10.1016/S1060-3743(03)00038-9

Chen, H.-J. H. (2009). Evaluating two web-based grammar checkers – Microsoft ESL Assistant and NTNU statistical grammar checker. Computational Linguistics and Chinese Language Processing, 14(2), 161–180.

Chodorow, M., Tetreault, J. R., & Han, N.-R. (2007). Detection of Grammatical Errors Involving Prepositions. In F. Costello, J. Kelleher and M. Volk (Eds.), Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions (pp. 25–30). https://doi.org/10.3115/1654629.1654635

De Felice, R., & Pulman, S. G. (2008). A classifier-based approach to preposition and determiner error correction in L2 English. In D. Scott and H. Uszkoreit (Eds.), Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) (pp. 169–176). https://doi.org/10.3115/1599081.1599103

Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006

Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1, 3–18. https://doi.org/10.5070/L2.V1I1.9054

Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and unfocused written corrective feedback in an English as a foreign language context. System, 36, 353–371. https://doi.org/10.1016/j.system.2008.02.001

Feng, H.-H., Saricaoglu, A., & Chukharev-Hudilainen, E. (2016). Automated error detection for developing grammar proficiency of ESL learners. CALICO Journal, 33(1), 49–70. https://doi.org/10.1558/cj.v33i1.26507

Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 81–104). Cambridge, England: Cambridge University Press. https://doi.org/10.1017/CBO9781139524742.007

Ferris, D. (2011). Treatment of error in second language writing (2nd ed.). Ann Arbor, MI: University of Michigan Press. https://doi.org/10.3998/mpub.2173290

Ferris, D., Liu, H., Sinha, A., & Senna, M. (2013). Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22, 307–329. https://doi.org/10.1016/j.jslw.2012.09.009

Gamon, M., Leacock, C., Brockett, C., Dolan, W. B., Gao, J., Belenko, D., & Klementiev, A. (2009). Using statistical techniques and web search to correct ESL errors. CALICO Journal, 26(3), 491–511. https://doi.org/10.1558/cj.v26i3.491-511

Granger, S. (2003). Error-tagged learner corpora and CALL: A promising synergy. CALICO Journal, 20(3), 465–480.

Guénette, D. (2007). Is feedback pedagogically correct? Research design issues in studies of feedback on writing. Journal of Second Language Writing, 16, 40–53. https://doi.org/10.1016/j.jslw.2007.01.001

Han, N.-R., Chodorow, M., & Leacock, C. (2006). Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12(2), 115–129. https://doi.org/10.1017/S1351324906004190

Hoang, G. T. L., & Kunnan, A. J. (2016). Automated essay evaluation for English language learners: A case study of MY Access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121

Jurma, W. E., & Deidre, L. F. (1984). Effects of immediate instructor feedback on group discussion participants. Central States Speech Journal, 35(3), 178–86. https://doi.org/10.1080/10510978409368186

Lavolette, E., Polio, C., & Kahng, J. (2014). The accuracy of computer-assisted feedback and students’ responses to it. Language Learning & Technology, 19(2), 50–68.

Liu, S., & Kunnan, A. J. (2016). Investigating the application of automated writing evaluation to Chinese undergraduate English majors: A case study of WriteToLearn. CALICO Journal, 33(1), 71–91.

Samuels, S. J., & Wu, Y. (2003). The effects of immediate feedback on reading achievement. Technical report. Minneapolis: University of Minnesota. Retrieved from http://www.epsteineducation.com/home/articles/file/research/immediate_feedback.pdf

Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on learners’ accuracy in using two English grammatical structures. Language Learning, 64, 103–131. https://doi.org/10.1111/lang.12029

Tetreault, J. R., & Chodorow, M. (2008). The ups and downs of preposition error detection in ESL writing. In D. Scott and H. Uszkoreit (Eds.), Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) (pp. 865–872). https://doi.org/10.3115/1599081.1599190

Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 16(4), 255–272. https://doi.org/10.1016/j.jslw.2007.06.003

Yi, X., Gao, J., & Dolan, W. B. (2008). A web-based English proofing system for English as a second language users. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP 2008) (pp. 619–624).