The National Core French Assessment Project/ Le projet pancanadien en évaluation en français, langue seconde: Development, field-testing and implementation of formative evaluation instruments

Larry Vandergrift and Claire Bélanger

 

Recent curriculum changes in core French (CF) programs across Canada resulted in the need for new evaluation tools. In response, the National Core French Assessment Project (Le projet pancanadien en évaluation en français de base) was established by the Canadian Association of Second Language Teachers (CASLT) to create new formative evaluation instruments. A variety of prototype instruments, congruent with the curriculum model advocated by the National Core French Study (NCFS), have been developed and tested for classroom use. The first phase of this three-year project focussed on evaluation at the intermediate level (Vandergrift & Bélanger, 1998). The second phase focussed on the beginning level, that is, CF instruction in the elementary school from Grades four to six (Vandergrift, in press). Finally, the third phase focussed on the development of instruments for students in CF programs in senior high school at the advanced level (Vandergrift, 1999).

Theoretical framework

Formative evaluation seeks to enhance learning by providing students with feedback on their progress in meeting the stated learning outcomes (Allal, 1991). In contrast to summative evaluation, which emphasizes student mastery of content or ranking of student performance, formative evaluation emphasizes a student's strengths and weaknesses and offers suggestions for improvements. The formative evaluation instruments created for all levels of this projected were expected to conform with the following principles (CASLT, 1996, p. 12-16):

1. Formative evaluation is most useful and valid when it is consistent with the teaching methodology (d'Anglejan et al., 1990); in this case, a communicative/experiential methodology as delineated by the NCFS.

2. In order to be valid, these formative evaluation instruments must be directly linked to common learning outcomes.

3. Formative evaluation is best conducted in the context of language tasks (Lussier, 1991).

4. Formative evaluation is most useful when it is criterion-referenced; i.e., assessing only a limited number of specific outcomes at one time (Belair, 1995).

5. Formative evaluation is most effective for improving performance when students are given quality feedback (Cornfield, et al. 1994).

6. Formative evaluation is more effective when students are progressively and systematically involved in the process (Harris, 1997; Scallon, 1996; Smolen et al., 1995).

7. Formative evaluation is more effective when diversified (Huerta-Macias, 1995).

8. Formative evaluation is preferably conducted in the target language.

Development and field-testing of instruments

Design teams of teachers from across the country, under the direction of project director Claire Bélanger, developed the instruments. The types of instruments included teacher checklists, student checklists, self-evaluations, peer evaluations, observation charts and rating scales. The specific challenge facing each team was to produce instruments that were task-based, reflective of a multidimensional curriculum and generic enough to be used across all fields of experience. Team members drew heavily upon their own teaching experience, recent documents on formative evaluation and recent literature on new evaluation practices (e.g., Allal, 1991; Bélair, 1995; Harris, 1997; Lussier, 1991; Lussier & Turner, 1995; McNamara & Dean, 1995; Wiggins, 1995).

To ensure validity, these instruments were field-tested by teachers and students across Canada. Teachers were asked to provide feedback on the technical aspects of the instrument and its usefulness for formative evaluation. They were asked if they would use the instrument again, and, most importantly, they were asked to comment on the capacity of the instrument to provide appropriate feedback and to realize positive effects on student learning. Finally, teachers were asked to identify the strong points and weak points of each instrument and to provide general reactions. Students were asked to comment on what they liked best about the task and instrument and how the instrument might help them to improve their performance in French.

Results

Beginner level

Teachers agreed (91%) that these instruments could have positive effects on student progress in learning French. Their responses indicated a high degree of satisfaction with these instruments as appropriate and useful evaluation tools; in fact, 93% of the participating teachers said that they would use these instruments again. Furthermore, teachers acknowledged 1) the adaptability of these instruments; 2) the potential for quality feedback to students; 3) the innovative ways in which student learning was enhanced; 4) the importance of reflection on learning in developing language learning strategies; and, 5) that task-based evaluation with these instruments made the evaluation process more enjoyable.

Student feedback was collected through a plenary discussion with the teacher. Students commented that 1) they enjoyed the authentic communication tasks; 2) these instruments pointed out what they did well, instead of only pointing out what was wrong; 3) the instruments were useful for improving their French; 4) the instruments helped them to organize their work; and, 5) they became more aware of the processes in language learning, particularly for listening and reading.

Intermediate level

Teacher reaction to the instruments at the intermediate level was very positive; 96% of the participating teachers indicated that they would use these kinds of instruments again. Teachers agreed that these instruments 1) will help evaluate student learning in relation to the targeted learning outcomes (86%); 2) do allow for appropriate feedback to students on their learning (87%); and, 3) can potentially have positive effects on student progress in learning French (91%). In their comments teachers noted that evaluation using these instruments is authentic, motivating, integrated, organized and precise, flexible and capable of fostering reflection on and autonomy in learning, but difficult for students to carry out in French.

Although student responses did not manifest the same level of positive reaction as teacher response, the overall reaction was positive. Over 60% of students indicated the feedback would be helpful in improving future performance in French. Most of the students who replied "partly" and "no" noted that the teacher had not made any comments that would help improve future performance. Furthermore, students commented that evaluation using these instruments 1) was motivating and confidence-building; 2) helped them feel more involved in their learning; 3) was diagnostic and directive for future learning; 4) helped them organize their work; but, 5) difficult to do in French.

Advanced level

All the teachers who tried a task and related instruments stated that the instruments had the potential for positive effects on student learning and all teachers affirmed that they would use the task and instruments again. Teachers commented on 1) the good curriculum fit and adaptability of the instruments to context and language level; 2) the integration of technology into many of the tasks; and, 3) the improved performance by their students.

Although they were not as positive as the teachers, students generally appreciated the learning experience and the usefulness of the assessment instruments. Their attitude to the assessment tools generally correlated with the degree of stated interest in, and involvement with the task. They commented on the 1) contextualized teaching of grammar (a welcome relief from regular grammar lessons) 2) personal comments by the teacher that give meaning to the checkmarks; 3) the potential of some instruments to stifle creativity; and 4) "...fun ways to practice French instead of just another writing assignment." The nature of communicative/experiential assessment, grounded in the principle of communicating authentic messages, appears to be deeply appreciated by students.

Discussion

Both student and teacher comments highlight the importance of experiential, task-based evaluation. While some of the enthusiasm might be attributed to the novelty of this kind of evaluation, the strong response demonstrates a genuine appreciation for active student involvement in learning. Students perceived these evaluation tasks as "real French," in contrast to what they normally did in class. They were engaged in experiential learning (i.e., authentic communicative activities) complemented by and interwoven with congruent assessment activities (i.e., formative evaluation activities that stimulate active involvement in the learning product and the process). This reinforced the teaching methodology and gave students a greater sense of empowerment. Furthermore, students were more involved in their learning through their active involvement in the evaluation process. Pearson and Berghoff (1996) note that the interweaving of assessment and instruction encourages students to participate actively in their own learning and leads to their empowerment.

Second, formative evaluation is motivational. Because of its diagnostic character, formative evaluation provided students with feedback that focussed on more than just what was wrong. Students learned what they did well and what they needed to improve. An emphasis on the process of learning as well as the creation of the language products (tasks) led students to be more motivated. This is consistent with Deci and Ryan (1985) who contend that, to the extent that teachers support autonomy and provide informative feedback, students’ sense of self-determination and intrinsic motivation can be enhanced. Involvement of students in learning through formative evaluation activities provides them with a sense of autonomy and feedback that both informs them of what they do well and where they can further improve. This leads language learners to approach a learning task positively; it represents a challenge to their existing competencies and requires them to use their creative capabilities.

Finally, these instruments made students more aware of the learning process and how to become more autonomous language learners. Both teachers and students commented on the power of these instruments to enhance learning; i.e., to plan for the successful completion of an oral or written production task or to reflect on the processes in comprehending an oral or a written text. Engaging students in reflection on their learning made them more aware of the learning processes and the strategies involved to become more successful, autonomous language learners. Students become more aware of the components of a task or the steps that lead to the successful completion of a task. As pointed out by Harris (1997), getting students to reflect on their own performance is perhaps the key to perceiving progress in communicative terms. Since progress in language learning in terms of communicative ability can often seem to be elusive, regular systematic self-assessment, in addition to other types of formative evaluation, can make students aware of gains in communicative ability that may not be perceptible otherwise.

Conclusion

The overall reaction to these formative evaluation instruments is very positive. Results indicate that these instruments have the potential to appropriately evaluate the targeted learning outcomes and to provide useful feedback to students on their performance. Teachers perceive these prototype instruments as adaptable, flexible tools that facilitate evaluation and learning. Comments from both students and teachers suggest that these instruments can 1) help students organize their work; 2) raise student awareness of the processes of language learning; 3) provide students with useful feedback; and, 4) motivate students to improve their performance in French.

The implementation of these instruments across Canada began in 1996 and continues today. Subsequent to each of the field-tests, necessary revisions were completed and the instruments were made available for use by FSL teachers in Canada and elsewhere (CASLT, 1996; 1998; 1999a). The provinces and territories have the responsiblility for the implementation of these instruments in schools and for in-servicing of teachers. However, in response to the need expressed by teachers involved in the field-tests, a video and accompanying handbook for workshop facilitators was developed by CASLT to guide teachers in the effective use of these formative evaluation instruments (CASLT, 1999b).

References

Allal, L. (1991). Vers une pratique de l'évaluation formative. Bruxelles: De Boeck-Wesmael.

Belair, L. (1995). Profil d'évaluation: Une analyse pour personnifier votre practique. Montréal, Qc: Les Éditions de la Chenelière.

Canadian Association of Second Language Teachers (CASLT). (1996). Évaluation formative des apprentissages en français, langue seconde/Formative Evaluation of Students in French as a Second Language: Niveau intermédiaire/Intermediate level. Ottawa, ON: CASLT.

Canadian Association of Second Language Teachers (CASLT). (1998). Évaluation formative des apprentissages en français, langue seconde/Formative Evaluation of Students in French as a Second Language: Niveau débutant/Beginner level. Ottawa, ON: CASLT.

Canadian Association of Second Language Teachers (CASLT). (1999a). Évaluation formative des apprentissages en français, langue seconde/Formative Evaluation of Students in French as a Second Language: Niveau avancé/Advanced level. Ottawa, ON: CASLT.

Canadian Association of Second Language Teachers (CASLT). (1999b). L’évaluation en classe de français de base/Classroom-Based Evaluation in Core French. Ottawa, ON: CASLT.

Cornfield, R. J., K. Coyle, B. Durrant, K. McCutcheon, J. Pollard & W. Stratton. (1987). Making the Grade. Montréal, QC: Éditions de la Chenelière.

D'Anglejan, A., B. Harley & S. Shapson. (1990). Student evaluation in a multidimensional core French curriculum. Canadian Modern Language Review, 47, 106-124.

Deci, E. L. & Ryan, R. M. (1995). Intrinsic motivation and self-determination in human behaviour. New York: Plenum.

Harris, M. (1997). Self-assessment of language learning in formal settings. ELT Journal, 51, 11- 20.

Huerta-Macias, A. (1995). Alternative assessment: Responses to commonly asked questions. TESOL Journal, 5(3), 8-11.

Lussier, D. (1991). L'évaluation selon une démarche communicative/expérientielle. Montréal, QC: Centre Éducatif et Culturel.

Lussier, D. & C. Turner. (1995). Évaluer les apprentissages dans une approche communicative. Montréal, Qc: Centre Éducatif et Culturel.

McNamara, M.J. & D. Deane. (1995). Self-assessment activities: Toward autonomy in language learning. TESOL Journal, 5(3), 17-21.

Pearson, B. & Berghoff, C. (1996). London Bridge is not falling down: It’s supporting alternative assessment. TESOL Journal, 6, 28-31.

Scallon, G. (1996). L'évaluation formative et le temps d'enseigner. Vie pédagogique, 99, 4-9.

Smolen, L., Newman, C., Wathen, T. & Lee, D. (1995). Developing student self-assessment strategies. TESOL Journal, 5(3), 22-27.

Vandergrift, L. (1995). Commonalities in core French curriculum documents in Canada. Edmonton, AB: Language Services Branch, Alberta Education.

Vandergrift, L. (in press). Setting students up for success: Formative evaluation and FLES. Foreign Language Annals.

Vandergrift, L. (1999). National Core French Assessment Project (Advanced Level): Report on the Field Test. Ottawa, ON: Canadian Association of Second Language Teachers.

Vandergrift, L. & Bélanger, C. (1998). The National Core French Assessment Project: Design and field test of formative evaluation instruments at the intermediate level. Canadian Modern Language Review, 54(4), 557-578.

Wiggins, G. (1995). Student-centered assessment. Presentation to the McGill University Center for Educational Leadership, Montreal.

 

About the Authors

Larry Vandergrift (Ph.D. University of Alberta) is professor in the Faculty of Education of the University of Ottawa. After a career of more than twenty years as a high school Core French teacher he now works in FSL teacher education in the B.Ed. program, as well as in the B.A. and M.Ed. programs in Second Language Teaching. Prior to coming to Ottawa he worked on the team that developed the new multidimensional Core French curriculum in Alberta. His research interests focus on listening comprehension, learning strategies and effective delivery of Core French programs in Canada.

Claire Bélanger est directrice du Projet pancanadien en évaluation. Chargée de projets pour les commissions scolaires et les ministères d'éducation, elle assure aussi des séminaires et des cours en didactique des langues secondes, plus particulièrement en curriculum, en enseignement et en évaluation. Ses publications couvrent les aspects pratiques du domaine. Elle compte vingt ans d'expérience comme enseignante et conseillère pédagogique de français de base et d'immersion française.

contents/sommaire