44 CHAPTER 3 OPERATIONALIZING EFFECTIVENESS Another challenge in designing and setting up the series of studies underlying this dissertation was the operationalization of e ectiveness. E ectiveness is generally measured in terms of (ultimate) language pro ciency attained by the students. Pro ciency, in turn, is usually measured through both receptive and productive tasks and tests. Within the language classroom, receptive skills are reading and listening, which in the Netherlands are traditionally measured using (closed type) listening and reading tests. Cito (the Dutch national testing agency) designs and validates tests that are compulsory for all schools at all levels (this is most poignantly illustrated in the reading tests that make up the nal exam for any modern foreign language in the Netherlands) or recommended but administered and used by almost all schools (e.g., listening skills as tested as part of ( nal) school examinations). When comparing the e ectiveness of receptive skills attained by DUB and SB students, it was therefore relatively easy to rely on these nationally administered, standardized tests. However, foreign language pro ciency is much more than merely attaining reading and listening skills. A pro cient learner, especially in a united Europe aiming at the integration of its citizens (European Parliament, 2022), should be able to converse and interact freely with other speakers (native and non-native alike) of that target language. erefore, it is even more interesting to measure e ectiveness of di erent teaching programs in terms of free oral and written production. A number of problems occur, however, when designing tests to measure productive skills. First, overreliance on more explicit measures of linguistic ability in testing may favor explicit types of instruction. Several authors (e.g., Doughty, 2003; DeKeyser, 2003) have cautioned against this possible test bias due to the di erential nature of instruction type (predominantly explicit vs. predominantly implicit; see Chapter 2 for a more elaborate discussion). Finding a free response task that (objectively) assesses speaking and writing pro ciency and that does not bias SB students was therefore a challenge. RECONCILING ECOLOGICAL VALIDITY AND EXPERIMENTAL CONTROL A third and nal challenge was obtaining ecological validity and at the same time ensuring a degree of experimental control. According to Hulstijn (1997), Second Language Acquisition (SLA) investigations tend to be primarily pursued under laboratory conditions, which is counterintuitive as ecologically valid L2-instruction by default takes place in real-life, actual classrooms. In a survey of SLA research, DeKeyser and Botano (2019) qualify the number of articles reporting on actual classroom experiments as “…distressingly small from the point of view of practitioners eager for research ndings that can unambiguously inform their classroom teaching…” (p. 4). Although experimental control is considered of the utmost importance in SLA research, we also considered it vital to design a study meeting ecological validity demands to
RkJQdWJsaXNoZXIy MTk4NDMw