Focus and Effects of Peer and Machine Feedback on Chinese University EFL Learners’ Revisions of English Argumentative Essays

Meihua Liu
https://orcid.org/0000-0002-5951-4167

Abstract

The present mixed-method study examined the foci and effects of peer and machine feedback on the revisions of Chinese university EFL learners’ English argumentative essays. The data included Draft 1, peer feedback (PF), PF-based Draft 2, machine feedback (MF), MF-based feedback, questionnaires, and interview recordings. The main findings were: (a) peer feedback was primarily concerned with content errors, while machine feedback mainly involved non-content errors, (b) significant differences occurred in errors of most types between Draft 1, PF and PF-based Draft 2, and between Draft 1, MF, and MF-based Draft 2, (c) the intake of ‘introducing a new topic in Conclusion’ was a powerful predictor of PF-based Draft 2 scores, and (d) the participants generally moderately considered peer and machine feedback to be useful. Based on the findings, some implications are discussed on how to better implement and enhance the quality of peer and machine feedback.


Keywords

peer feedback; machine feedback; focus; effect; rewrite; argumentative essay

AbuSeileek, A., & Abualsha’r, A. (2014). Using peer computer-mediated corrective feedback to support EFL learners’ writing. Language Learning & Technology, 18(1), 76–95. http://llt.msu.edu/issues/february2014/abuseileekabualshar.pdf

Allen, D., & Katayama, A. (2016). Relative second language proficiency and the giving and receiving of written peer feedback. System, 56(1), 96–106.

Anson, C. M. (2006). Can’t touch this: Reflections on the servitude of computers as readers. Machine scoring of student essays. In P. Freitag Ericsson & R. Haswell (Eds.), Machine scoring of student essays (pp. 38–56). Logan, Utah: Utah State University Press.

Bijami, M., Kashef, S. H., & Nejad, M. S. (2013). Peer feedback in learning English writing: Advantages and disadvantages. Journal of Studies in Education, 3(4), 91–97. https://doi.org/10.5296/jse.v3i4.4314

Caulk, N. (1994). Comparing teacher and student responses to written work. TESOL Quarterly, 28(1), 181–188.

Charmaz, K. (2006). Constructing grounded theory. London: Sage.

Chen, C. E., & Cheng, W. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning and Technology, 12(2), 94–112.

Cheng, G. (2017). The impact of online automated feedback on students’ reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18–27.

Cho, K., & Schunn, C. D. (2005). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education, 48, 409–426. https://doi.org/10.1016/j.compedu.2005.02.004

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Cope, B., Kalantzis, M., McCarthey, S., Vojak, C., & Kline, S. (2011). Technology-mediated writing assessments: Principles and processes. Computers and Composition, 28, 79–96.

Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17.

Dochy, F., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: a review. Studies in Higher Education, 24, 331–350.

Ellis, M. J. (2011). Peer feedback on writing: Is on-line actually better than on-paper? Journal of Academic Language & Learning, 5(1), A88–A99.

Ferris, D. (2010). Second language writing research and written corrective feedback in SLA: Intersections and practical applications. Studies in Second Language Acquisition, 32, 191–201.

Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20, 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007

Goldstein, L. M. (2004). Questions and answers about teacher written commentary and student revision: Teachers and students working together. Journal of Second Language Writing, 13(1), 63–80. http://dx.doi.org/10.1016/j.jslw.2004.04.006

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39, 83–101.

Ion, G., Barrera-Corominas, A., & Tomàs-Folch, M. (2016). Written peer-feedback to enhance students’ current and future learning. International Journal of Educational Technology in High Education, 13(15). https://doi.org/10.1186/s41239-016-0017-y

Keh, C. L. (1990). Feedback in the writing process: a model and methods for implementation. ELT Journal, 44(4), 294–304.

Kramer, M. G., Leggett, G., & Mead, D. (1995). Prentice hall handbook for writers. Englewood Cliffs, NJ: Prentice Hall.

Kulkarni, C., Bernstein, M. S., & Klemmer, S. (2015). Peer studio: Rapid peer feedback emphasizes revision and improves performance. Behavior, 2(1), 14–18.

Kurt, G., & Atay, D. (2007). The effects of peer feedback on the writing anxiety of prospective Turkish teachers of EFL. Journal of Theory and Practice in Education, 3(1), 12–23. http://eku.comu.edu.tr/index/3/1/gkurt_datay.pdf

Li, J, Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18.

Lin, W.-H., & Yang, S. C. (2011). Exploring students’ perceptions of integrating Wiki technology and peer feedback into English writing courses. English Teaching: Practice and Critique, 10(2), 88–103. http://education.waikato.ac.nz/research/files/etpc/files/2011v10n2dial1.pdf

Liu, M., & Chai, Y. (2009). Attitudes towards peer review and reaction to peer feedback in Chinese EFL writing classrooms. TESL Reporter, 40(1), 33–50.

Liu, N. F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279–290.

Lu, R., & Bol, L. (2007). A Comparison of anonymous versus identifiable e-Peer review on college student writing performance and the extent of critical feedback. Journal of Interactive Online Learning, 6(2), 100–115.

Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18, 30–43. https://doi.org/10.1016/j.jslw.2008.06.002.

Miao, Y., Badger, R., & Zhen, Y. (2006). A comparative study of peer and teacher feedback in a Chinese EFL writing class. Journal of Second Language Writing, 15, 179–200.

Min, H. T. (2006). The effects of trained peer review on EFL students’ revision types and writing quality. Journal of Second Language Writing, 15, 118–141. https://doi.org/10.1016/j.jslw.2006.01.003.

Nobles, S., & Paganucci, L. (2015). Do digital writing tools deliver? Student perceptions of writing quality using digital tools and online writing environments. Computers and Composition, 38, 16–31.

Philips, S. M. (2007). Automated essay scoring: A literature review (SAEE research series #30). Kelowna, BC: Society for the Advancement of Excellence Education.

Rollinson, P. (1998). Peer response and revision in an ESL writing group: A case study. Universidad Autónoma de Madrid, Unpublished PhD thesis.

Rollinson, P. (2005). Using peer feedback in the ESL writing class. ELT Journal, 59(1), 23–30. https://doi.org/10.1093/elt/cci003.

Shermis, M. D., & Burstein, J. (Eds.). (2003). Automated essay scoring: A cross-disciplinary perspective. Hillsdale, NJ: Lawrence Erlbaum Associates.

Shintani, N. (2015). The effects of computer-mediated synchronous and asynchronous direct corrective feedback on writing: A case study. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2014.993400.

Stanley, J. (1992). Coaching student writers to be effective peer evaluators. Journal of Second Language Writing, 1, 217–233.

Sterna, L. A., & Solomo, A. (2006). Effective faculty feedback: The road less traveled. Assessing Writing, 11, 22–41.

Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65.

Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249–276. http://www.jstor.org/stable/1170598

Ullmann, T. D. (2019). Automated analysis of reflection in writing: Validating machine learning approaches. International Journal of Artificial Intelligence in Education, 29, 217–257. https://doi.org/10.1007/s40593-019-00174-2

Vardi, I. (2009). The relationship between feedback and change in tertiary student writing in the disciplines. International Journal of Teaching and Learning in Higher Education, 20(3), 350–361. http://www.isetl.org/ijtlhe

Wang, W. (2014). Students’ perceptions of rubric-referenced peer feedback on EFL writing: A longitudinal inquiry. Assessing Writing, 19, 80–96. http://dx.doi.org/10.1016/j.asw.2013.11.008

Warschauer, M., & Ware, J. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10(2), 1–24.

Winne, P. H., & Butler, D. L. (1994). Student cognition in learning from teaching. In T. Husen & T. Postlewaite (Eds.), International encyclopedia of education (2nd ed.) (pp. 5738–5745). Oxford, UK: Pergamon.

Wyrick, J. (2008). Steps to writing well. Beijing: Peking University Press.

Xie, Y., Ke, F., & Sharma, P. (2008). The effect of peer feedback for blogging on college students’ reflective learning processes. Internet and Higher Education, 11, 18–25. https://doi.org/10.1016/j.iheduc.2007.11.001.

Yu, S. & Lee, I. (2015). Understanding EFL students’ participation in group peer feedback of L2 writing: A case study from an activity theory perspective. Language Teaching Research, 19(5), 572–593.

Download

Published : 2021-01-29


LiuM. (2021). Focus and Effects of Peer and Machine Feedback on Chinese University EFL Learners’ Revisions of English Argumentative Essays. Theory and Practice of Second Language Acquisition, 7(1), 75-98. https://doi.org/10.31261/TAPSLA.8642

Meihua Liu 
Tsinghua University  China
https://orcid.org/0000-0002-5951-4167




Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The Copyright Holders of the submitted texts are the Authors. The Reader is granted the rights to use the material available in the TAPSLA websites and pdf documents under the provisions of the Creative Commons 4.0 International License: Attribution - Share Alike  (CC BY-SA 4.0). The user is free to copy and redistribute the material in any medium or format, and to remix, transform, and build upon the material for any purpose, even commercially.

1. License

The University of Silesia Press provides immediate open access to journal’s content under the Creative Commons BY-SA 4.0 license (http://creativecommons.org/licenses/by-sa/4.0/). Authors who publish with this journal retain all copyrights and agree to the terms of the above-mentioned CC BY-SA 4.0 license.

2. Author’s Warranties

The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s.

If the article contains illustrative material (drawings, photos, graphs, maps), the author declares that the said works are of his authorship, they do not infringe the rights of the third party (including personal rights, i.a. the authorization to reproduce physical likeness) and the author holds exclusive proprietary copyrights. The author publishes the above works as part of the article under the licence "Creative Commons Attribution-ShareAlike 4.0 International".

ATTENTION! When the legal situation of the illustrative material has not been determined and the necessary consent has not been granted by the proprietary copyrights holders, the submitted material will not be accepted for editorial process. At the same time the author takes full responsibility for providing false data (this also regards covering the costs incurred by the University of Silesia Press and financial claims of the third party).

3. User Rights

Under the CC BY-SA 4.0 license, the users are free to share (copy, distribute and transmit the contribution) and adapt (remix, transform, and build upon the material) the article for any purpose, provided they attribute the contribution in the manner specified by the author or licensor.

4. Co-Authorship

If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.

I hereby declare that in the event of withdrawal of the text from the publishing process or submitting it to another publisher without agreement from the editorial office, I agree to cover all costs incurred by the University of Silesia in connection with my application.