CHAPTER 5. Writing skills 77 language used. Free response writing assignments were given to the students to enable them to practice for the assignment used in the current study. TESTING PROCEDURE e writing test for both cohorts took place in Year 6. e teacher selected four topics for students to prepare and during the test, students were o ered two of those topics and they were asked to write an essay of a minimum of 200 words on one of these topics within 50 minutes (See Appendix C for an example of such a writing exam). e students wrote the essay in the computer lab at their school and handed them in digitally. Supportive tools (e.g., spelling- and grammar checker) were not turned o during the test, but they were hardly used given the time restrictions (the teacher was able to monitor all screens by using specialized so ware). e anonymized essays from both cohorts were assessed at the same time through holistic ratings by expert teachers, machine-mediated morphosyntactic pro ling and by means of analytical Complexity, Accuracy and Fluency (CAF) measures (see below). HOLISTIC RATINGS BY EXPERT TEACHERS To rate the texts holistically, the same method was used as in Verspoor et al. (2012). A group of 9 French teachers were asked to rank 10 texts in terms of general pro ciency in several rounds until consensus was reached. ese texts were used as benchmarks and rubrics were created to describe the benchmarked texts (see Appendix A). A er this ve-hour session, the nine raters were divided into three groups of three raters each. e texts were divided in 4 batches of 12 and a nal batch of 8. To avoid bias, the raters were randomly divided into new groups with each new batch of essays. Agreement among the raters was high. In SPSS (version 27), the Intraclass Correlation Coe cient (two way random, consistency with a 95% con dence interval) on the ratings produced by three groups of three independent raters was r=0.893 (p<.001). MACHINE-MEDIATED MORPHOSYNTACTIC PROFILING Direkt Profil (DP) is one of the few corpus tools available for FL French and was developed at the University of Lund in Sweden (Granfeldt et al., 2006)7. In total, the software bases its analyses on 142 different analytic text measures in profiling the morphosyntactic content of any (learner) text including subject-verb agreement, tense use, number of conjunctions, etc. Three different algorithms (Granfeldt et al., 2006) produce a profile based on the six morphosyntactic stages of development, from 7 As there were no funds for further development and the computer language used for the program was outdated, the so ware had to be removed from the server for security reasons and is no longer available online since 2021.
RkJQdWJsaXNoZXIy MTk4NDMw