OISE/UT Bulletin 2000/2001 -- University of Toronto Graduate Studies in Education
Curriculum, Teaching and Learning - Measurement and Evaluation Program
Search the Bulletin for a word or phrase:

MEASUREMENT AND EVALUATION PROGRAM

Program Coordinator:

R.E. Traub

Core Faculty

M.-J. Berger
R. Childs
G. Hanna
T.C. M. Lam
P. Nagy
S. Nishisato
M.W. Wahlstrom
R.G. Wolfe

Associate Faculty

S. Churchill

The Measurement and Evaluation Program is designed to develop knowledge and skills in quantitative and qualitative methods used in research and in evaluation. For the Master’s Degree, the domains covered include the design of research and evaluation studies, the development of instruments and related theory, the collection and analysis of data, and statistical theories and methods. For the Doctoral degree, students are expected to specialize in either Measurement or Evaluation. The Measurement specialization is for those interested in quantitative methods and theory, including psychometric and statistical theory. The Evaluation specialization is for those interested in the evaluation of programs (including a wide range of social and educational programs) or in student evaluation (including large-scale assessments of student achievement), using both quantitative and qualitative methodologies.

Master of Education

Applicants are accepted under the SGS general regulations which specify an appropriate four-year University of Toronto bachelor's degree, or its equivalent from a recognized university, completed with standing equivalent to a University of Toronto mid-B or better in the final year. Applicants with an appropriate three-year University of Toronto bachelor’s degree or its equivalent may also be accepted. Ordinarily, applicants will have at least one year of relevant, successful, professional experience prior to applying. The M.Ed. program may be undertaken on a full or part-time basis, and consists of eight half-courses plus a research project/paper.

Candidates are ordinarily required to take courses CTL2006H, CTL2007H, CTL2008H, CTL2009H, and CTL2808H. Students who have not studied statistics at least to the level of course CTL2004H must take CTL2004H as an extra course. Additional study may be required either within the degree program or prior to admission. All requirements for the degree must be completed within six calendar years from first enrolment. (See the Minimum Admission, Program and Degree Requirements section for program requirements, pages 26 - 34)

In the Statement of Intent, applicants should site the reasons they wish to study measurement and evaluation at the graduate level. The chief academic interests, professional concerns, and career plans within measurement and evaluation should be discussed. The committee reviews this Statement to determine the area of study in which an applicant is most interested.

Master of Arts

Applicants are accepted under the SGS general regulations. The usual admission requirement for the M.A. is a four-year bachelor's degree in mathematics, statistics, computer science, or a relevant social or behavioral science. Ordinarily, applicants will have at least one year of relevant, successful, professional experience prior to applying. Students who anticipate going on to further study at the Ph.D. level are advised to apply for enrolment in an M.A. program rather than an M.Ed. program. The M.A. program may be undertaken on a full-time or part-time basis and consists of six to eight half-courses and a thesis. Candidates are ordinarily required to take courses CTL2006H, CTL2007H, CTL2008H, CTL2009H, and CTL2808H. Students who have not studied statistics at least to the level of course CTL2004H must take CTL2004H as an extra course. Additional courses may be required of some candidates. All requirements for the degree must be completed within six calendar years from first enrolment. (See the Minimum Admission, Program and Degree Requirements section for program requirements, pages 26 - 34)

In the Statement of Intent, applicants should site the reasons they wish to study measurement and evaluation at the graduate level. The chief academic interests, professional concerns, and career plans within measurement and evaluation should be discussed. The committee reviews this Statement to determine the area of study in which an applicant is most interested.

 

Doctor of Education

The Ed.D. program is designed for practicing social service professionals who wish to acquire knowledge and skills at the doctoral level in measurement and evaluation, with emphasis on applying these to research and practice. Applicants will ordinarily have a minimum of three years professional experience in education prior to application. A University of Toronto M.Ed. or M.A. in education or its equivalent from a recognized university, in the same area of specialization proposed at the doctoral level, completed with an average grade equivalent to a University of Toronto B+ or better is required. Admittance is contingent upon satisfactory completion of a Qualifying Research Paper (QRP) or a master’s thesis. An applicant’s admission will be confirmed, however, only when the QRP or master’s thesis is judged to be of sufficiently high quality to warrant admission.

The Ed.D. program normally consists of eight half-courses. Candidates are expected to have taken courses CTL2006H, CTL2007H, CTL2008H, CTL2009H, and CTL2808H or their equivalents. Students lacking one or more of these courses are required to take them in addition to the regular doctoral program, which must include at least four CTL2800-level courses. Students must successfully complete a comprehensive examination. A thesis embodying the results of an original investigation, and a final oral examination on the content and implications of the thesis are also required. The ideal arrangement is for Ed.D. applicants to begin their program on a part-time basis. All requirements for the degree must be completed within six calendar years from first enrolment. (See the Minimum Admission, Program and Degree Requirements section for program requirements, pages 26 - 34)

In the Statement of Intent, applicants should site the reasons they wish to study measurement and evaluation at the graduate level. The chief academic interests, professional concerns, and career plans within measurement and evaluation should be discussed. The committee reviews this Statement to determine the area of study in which an applicant is most interested.

Doctor of Philosophy

The Ph.D. program emphasizes theoretical concerns of the fields of measurement and evaluation. Applicants will ordinarily have a minimum of two years professional experience in education prior to applying. A University of Toronto M.A. in education or its equivalent from a recognized university, in the same area of specialization proposed at the doctoral level, completed with an average grade equivalent to a University of Toronto B+ or better is required. Admittance is contingent upon satisfactory completion of a Qualifying Research Paper (QRP) or a master’s thesis. An applicant’s admission will be confirmed, however, only when the QRP or master’s thesis is judged to be of sufficiently high quality to warrant admission.

The Ph.D. program normally consists of six half-courses. Candidates are expected to have taken courses CTL2006H, CTL2007H, CTL2008H, CTL2009H, and CTL2808H or their equivalents. Students lacking one or more of these courses are required to take them in addition to the regular doctoral program, which must include at least four CTL2800-level courses in addition to CTL2808. Students must successfully complete a comprehensive examination. A thesis embodying the results of an original investigation, and a final oral examination on the content and implications of the thesis are also required. The Ph.D. program must be initiated on a full-time basis and requires two years of consecutive full-time study. All requirements for the degree must be completed within six calendar years from first enrolment. (See the Minimum Admission, Program and Degree Requirements section for program requirements, pages 26 - 34)

In the Statement of Intent, applicants should site the reasons they wish to study measurement and evaluation at the graduate level. The chief academic interests, professional concerns, and career plans within measurement and evaluation should be discussed. The committee reviews this Statement to determine the area of study in which an applicant is most interested

COURSES

The following list demonstrates the range of courses offered within the Measurement and Evaluation program. Not all of the courses listed are offered in any given year.

CTL2000H Descriptive Statistics (Non-Credit)
Students enrolling in CTL2004, CTL2008, or other similar courses who have no background knowledge of elementary statistics are advised to take this non-credit course, or to make arrangements to acquire this knowledge.
Staff

CTL2001H Elementary Matrix Algebra (Non-Credit)
Introduction to vectors and matrices, basic operations, and such concepts as inverse, determinant, and singularity.
S. Nishisato

CTL2002H Alternatives to Quantitative Evaluation in Education
This course presents qualitative, contextually oriented alternatives to existing conventional, preordinately oriented strategies. Discussion focuses on different metaphors for evaluation, such as educational criticism, investigative journalism, jurisprudence, and naturalistic inquiry. Students will integrate theoretical ideas with fieldwork.
Staff

CTL2003H Measurement and Accountability
An examination of principles of measurement and their relationship to major issues in accountability, such as the validity of comparative international and national studies, the influence of external examination, and the effects of large-scale testing on students and the curriculum. The course is intended for both Measurement students and others.
P. Nagy

CTL2004H Elements of Statistics
Techniques for data exploration and description, basic theory behind hypothesis testing and confidence intervals, and common confirmatory techniques (correlation, simple linear regression, one-way analysis of variance). Students are expected to learn to use the MINITAB statistics package or equivalent. Prerequisite: Knowledge of elementary arithmetic and algebra.
G. Hanna, P. Nagy, S. Nishisato, R.E. Traub or staff

CTL2005H Applied Dual Scaling (Pass/Fail)
Application-oriented introduction to dual scaling and its rationale, with illustrations, for optimal quantification of different types of qualitative data, practicum in data analysis, interpretation of the scaling outcomes, and class presentations. This course is intended for both Measurement students and others.
S. Nishisato

CTL2006H Evaluation Models and Methods
How and why we evaluate social and educational programs is the major focus of this course: negotiation of the evaluation agreement, practical design considerations, timing, budget, and preparation of the final report. Qualitative, quantitative, and mixed approaches are studied and compared, including focus groups, case studies, quasi-experimental designs, and randomized field trials. An introduction to personnel evaluation, including teacher evaluation, is also included.
T. Lam or staff

CTL2007H Research Methods in Education
Basic concepts, methods, and problems in educational research are considered: discovering the periodicals in one's field, steps in the research process, developing research questions, design of instruments, methods of data collection and analysis, interpreting results, and writing research reports.
T. Lam or staff

CTL2008H Data Analysis in Education
An overview of widely used methods of data analysis, not covered in other 2000-level courses. Main topics are principal component analysis, factor analysis, cluster analysis, loglinear analysis and scaling (e.g. Rasch-type, Thurstone-type, and a variety of multidimensional scaling).
Prerequisite: Concurrent registration in CTL2001 (non-credit) or permission of the instructor.
S. Nishisato. R.G. Wolfe or staff

CTL2009H Instrument Development in Education
An overview of kinds of instruments used for collecting educational data. Classroom tests and various item types; norm-referenced and criterion-referenced standardized achievement tests; group intelligence and aptitude tests; attitude and self-report scales; observational systems, including performance assessment and classroom observation; questionnaires and surveys; interview protocols; reliability theory and item anlysis; and validity. The course will focus on selection criteria for commercially available instruments, and on criteria for use in refining researcher-designed instruments.
R.E. Traub or staff

CTL2010H Alternative Student Assessment Methods
The course examines the soundness and effects of a variety of alternative student assessments, such as authentic assessment, self and peer assessments, portfolio assessment, performance assessment, and individualized assessment, that have been developed for student-centered instructional and learning strategies. The course addresses research studies, and identifies research needs in using these assessment methods for both classroom and large-scale assessment purposes.
T. Lam

CTL2011H Path Analysis
This course is designed for graduate students who do not specialize in statistics but who need a significant familiarity with those statistical techniques known collectively as path analysis, Structural equation modeling, causal modeling or analysis of covariance structures. The primary objective of this course is to give students (1) the ability to recognize situations where these techniques may be useful in research; (2) an appreciation for the roles of sound theory and sound measurement in making these techniques useful; (3) an understanding of the limitations of these methods; (4) the ability to use available software in conducting research; and (5) the ability to understand published research using these techniques. LISREL software with the intuitive Simple Lisrel (SIMPLIS) software will be used. The course is open to graduate students from all departments. (Desirable prerequisite: undergraduate course in statistics or CTL1666.)
G. Hanna

CTL2797H Practicum in Measurement and Evaluation: Master's Level
Supervised experience in an area of fieldwork, under the direction of faculty and field personnel. Inquire at the department office at least two weeks before the beginning of term.
Staff

CTL2798H Individual Reading and Research in Measurement and Evaluation: Master's Level
Specialized study, under the direction of a staff member, focusing on topics of particular interest to the student. While course credit is not given for a thesis investigation proper, the study may be closely related to a thesis topic.
Staff

CTL2799H Special Topics in Measurement and Evaluation: Master's Level
A course designed to permit the study (in a formal class setting) of specific areas of measurement and evaluation not already covered in the courses listed for the current year. (This course does not fulfil the purpose of CTL2798, which is normally conducted on a tutorial basis.)
Staff

CTL2800H Structural Equations with Latent Variables
Beginning with basic tests of hypotheses on the structure of covariance matrices, the course leads to the standard problems of estimation and hypothesis testing for structural coefficients and for overall model fit. It also includes consideration of more general recursive and non-recursive path analysis models.
Desired background: CTL2001 (non-credit), CTL2808 or equivalents.
G. Hanna

CTL2801H Test Theory
Seminar on recent advances in test theory. Topics include classical test theory, generalizability theory, and item response theory, with applications to educational research problems.
Desirable background: CTL2001 (non credit), CTL2006, CTL2007, CTL2008, CTL2009, and CTL2808, or equivalents.
R.E. Traub

CTL2802H Performance Assessment
This course examines the conceptualization and implementation of performance assessment in the classroom, in large scale assessment programs, and in program evaluation, and addresses the technical issues that arise in the process. Topics addressed include scoring rubric construction, rater consistency and moderation, generalizability, sources of invalidity and bias, judgmental and empirical methods of analysing bias, comparability, standard setting, and logistics and cost effectiveness.
Prerequisites: CTL2009 or equivalent, or permission of instructor.
T. Lam

CTL2803H Evaluation Methodology and Theory
Theories and their links to methodology are used to examine the logic of evaluative inquiry. Subject matter theories, such as language theory, are seen to play a role as well as subjectivist and objectivist epistemologies. Prominent evaluation models (e.g., CIPP, illuminative, responsive) are compared and contrasted to bring out their conceptual bases. Students wishing an introduction to evaluation methodology should register for course CTL2006 and/or more specialized courses such as CTL2002, CTL1501, or AEC1133. Courses CTL2803 and CTL2006 are normally required for the Evaluation specialization.
Prerequisite: Master's-level course such as those listed above, or permission of instructor.
T. Lam

CTL2804H Measurement and Cognition
This course examines recent cognitive science and measurement literature on the investigation and assessment of complex learning outcomes such as problem solving, written production, and thinking skills.
Prerequisite: CTL2004 and CTL2008, or equivalents, or permission of the instructor.
P. Nagy

CTL2805H Scaling Theory
Detailed examination of dual scaling, advanced treatment of the Thurstonian methods of unidimensional scaling, several versions of both metric and nonmetric multidimensional scaling procedures.
Prerequisites: CTL2001 (Non-Credit) and either CTL2005 or CTL2008, or permission of instructor.
S. Nishisato

CTL2806H Evaluation of Educational Policies
Case studies including Third World settings illustrate use of qualitative and quantitative data in the context of organizational theory. The seminar emphasizes gathering and interpreting data from multiple sources (qualitative and quantitative). Intended for students interested in master's and doctoral research leading to recommendations about policy, curriculum reform, teacher training and/or resource allocation at any level: within individual schools, districts (school boards), regions, provinces/states or nations, as well as within colleges, universities and ministries.
S. Churchill

CTL2807H Applied Research Design and Data Analysis
This seminar is intended primarily for doctoral students. The are two main activities. One is the cooperative critiquing and development of research designs and data analysis plans based on ongoing work of the students in the course. The second is discussion of selected topics in data analysis, e.g. exploratory analysis, logistic regression, multilevel modeling, and recent advances in some of the techniques introduced in course CTL2008 and CTL2808.
Prerequisite: CTL2808, or permission of the instructor.
P. Nagy

CTL2808H Intermediate Statistics and Research Design
Analysis of variance for one-way and multi-way classification data for fixed, mixed, and random effects models and various designs (e.g., crossed, nested, composite, repeated measurements); multiple comparison tests (Tukey, Scheffé); linear and multiple regression; multiple correlation; analysis of covariance.
Prerequisite: CTL2004 or equivalent, or permission of instructors.
G. Hanna, P. Nagy, S. Nishisato, R. Traub or staff

CTL2809H Multivariate Analysis with Applications
Multivariate statistical procedures, including multiple regression analysis, multivariate significance tests, factor analysis, discriminant analysis, canonical analysis, multivariate analysis of variance, and general linear models, are discussed with application to data.
Prerequisite: CTL2001 (Non-Credit), CTL2008, CTL2808, or permission of instructors.
S. Nishisato or R.G. Wolfe

CTL2810H Seminar in Evaluation Problems
A seminar dealing with theories and practical constraints in the implementation of evaluation strategies in field settings.
Prerequisite: CTL2803 or equivalent.
Note: Practicum CTL2997 and Seminar CTL2810 may not both be taken for credit in fulfilling the requirements of the eight half-courses in the Ed.D. program in Evaluation.
Staff

CTL2811H Seminar in Measurement and Experimental Design
Consideration of theoretical and empirical topics in measurement, and a detailed examination of experimental designs relevant to educational research.
Prerequisite: Permission of the instructor.
Staff

CTL2997H Practicum in Measurement and Evaluation: Doctoral Level
Supervised experience in an area of fieldwork, under the direction of faculty and field personnel. Examples of typical activities are developing an evaluation project and/or acting as research assistant and advisor to educational bodies or agencies. Arrangements for this practicum are made through the Practicum Coordinator. Inquire at department office at least two months before beginning of term.
Staff

CTL2998H Individual Reading and Research in Measurement and Evaluation: Doctoral Level
Specialized study, under the direction of a staff member, focusing on topics of particular interest to the student. While course credit is not given for a thesis investigation proper, the study may be closely related to a thesis topic.
Staff

CTL2999H Special Topics in Measurement and Evaluation: Doctoral Level
A course designed to permit the study, (in a formal class setting) of specific areas of measurement and evaluation not already covered in the courses listed for the current year. (This course does not fulfil the purpose of CTL2998, which is normally conducted on a tutorial basis.)
Staff

OISE/UT Bulletin 2000/2001 -- University of Toronto Graduate Studies in Education
Search the Bulletin for a word or phrase: