Section JKL Mondays 1:00 – 3:50 Room TH241

Instructor:	Robert Runt’		Office: TH313		Phone: 403-329-2454
E-mail: Runte@uleth.ca				Fax: 403-329-2252 (always label ‘Attn: Dr. Runt’)’
Please Note: all emails to instructor must include Ed3504 in subject line to avoid automatic deletion by anti-spam software. Emails are read during office hours and every effort is made to answer within 3 working days.
Instructor website:		http://www.uleth.ca/edu/runte/
Administrative Support:		Margaret Beintema Phone 403-329-2732 Office: TH321

Course Description

Calendar Description: Introduction to a variety of approaches to evaluating student learning.

This is the first of two evaluation courses you will take as part of your professional training. Both courses emphasize the practical skills and knowledge you will require to become a successful classroom teacher.

This course provides a general introduction to evaluation issues, terminology, and principles. We will also be exploring a number of non-test assessment techniques, such as: performance-based assessment; observation checklists; rubrics; anecdotal records; learning logs and journals; portfolios; student and peer evaluation; and self-evaluation. Discussion of evaluation techniques such as the multiple-choice questions and essay tests will be left until Professional Semester II when student teachers will be taking responsibility for teaching entire units in their practicum, including the construction of unit tests and essay assignments.

Course Goals

By the end of the course, students will be able to:

speak knowledgeably about trends and issues in evaluation, such as the on-going debate over standardized tests, and the differences between assessment of learning, assessment for learning, and assessment as learning

understand and appreciate the role and importance of evaluation in monitoring one’s own teaching effectiveness

understand the difference between formative, summative, placement, and diagnostic evaluation, and use appropriate evaluation strategies in each context

define criterion-referenced and norm referenced evaluation, and interpret student results in the appropriate context

define and distinguish between validity and reliability

know and apply Bloom’s Taxonomy of the Cognitive Domain

design learning outcomes for individual lessons (e.g., write learning objectives)

relate objectives and evaluation to the provincial curriculum

identify potential sources of error and bias in alternative assessments, and how to address these

list the advantages and disadvantages of each (non-test) evaluation strategy

recall and apply basic principles of observational techniques

recall and apply basic principles of performance based assessment

recall and apply appropriate techniques in recording anecdotal records (using episode, performance, & time sampling)

design and use a checklist and rubric

identify and apply appropriate rating scales

design and manage learning logs, blogs, and journals

design appropriate assessment strategies using portfolios

identify appropriate strategies for a teacher/parent/student conference

demonstrate appropriate oral questioning techniques in the practicum

incorporate evaluation in their daily lesson planning and instruction in the practicum

Topic Outline

I. Evaluation Issues

1. The Who, What, When, and Why of Evaluation

2. Trends and Issues in Evaluation: Accountability, Standardized

Testing, and Alternative / Authentic Assessment

3. Assessment of learning vs. Assessment for learning vs. Assessment as

Learning

II. The Language of Evaluation

1. Basic Terms

A. Use: Placement, Formative, Diagnostic, Summative

B. Interpretation: Criterion vs. Norm reference

2. Bloom’s Taxonomy

III. Curriculum and Evaluation

1. Alberta Curriculum and Evaluation

2. The Diagnostic Teaching Model

3. Writing Specific Learning Objectives

IV. Criteria for Effective Evaluation

1. Basic Principles of Systematic Evaluation

2. Introduction to Validity and Reliability

3. Ethical Issues in Evaluation: Principles For Fair Student Assessment Practices

For Education In Canada

V. Evaluation Techniques (The ‘How To’ of Evaluation)

1. Oral Questioning and Class Discussion as Evaluation

2. Deskwork

3. Authentic Assessment

A. Basic Principles

B. Performance Based Assessment

4. Observational Techniques

A. Basic Principles

B. Anecdotal Records (Episode, Performance, & Time Sampling)

C. Checklists

D. Rating Scales

E. Worthy Tasks Rubrics

F. Potential Sources of Error

5. Learning Logs and Journals

6. Portfolios

VI. The Teacher/Parent/Student Conference (time permitting)

Tentative Schedule

Class

Date

Topics

Readings

Sept 14

Intro Role Play: How NOT to evaluate/teach

Why we evaluate

Intro (pp. x-xviii, xx-xxv)

Chapter 2 (pp. 26-36)

Sept 21

Basic Terms and Concepts:

Placement, Formative, Diagnostic, & Summative evaluation; Criterion vs. Norm reference;

3 Domains; & Bloom’s Taxonomy; validity & reliability; Assessment of learning vs. Assessment for learning vs. Assessment as Learning

Role of Learning Objectives in Evaluation

Role & importance of evaluation in monitoring one’s own teaching effectiveness

[Declare topic & mode for ‘In depth’ assignment]

Intro (pp. xviii –xx)

Chapter 2 (pp. 24-26)

Conclusion (203-205)

Chapter 3 (37-53)

Sept 28

Observation

Overview of Authentic Assessment

Chapter 1 (1-22)

Chapter 9 (147-162)

Oct 5

Performance Based Assessment

Checklists

Rubrics

Chapter 5 (75-102)

Oct 19

Oral Questioning (& metacognition)

[Checklist/Rubric assignment due]

Chapter 8 (133-146); Chapter 10 (154-162)

Oct 26

Portfolios/conferencing

Journals and learning logs

Chapter 4 (55-74)

Chapter 11 (173-184)

Chapter 7 (119-132)

Nov 2

Principles of Fair & Accurate Assessment

[‘In depth’ Assignment Due]

Principles document

(on-line download)

Chapter 12 (185-202)

Nov 9

[Course Examination]

All of the above

Required Texts

Grading

Excellent

Good

Satisfactory*

Poor**

Failing

96 - 100 A+

90 - 95 A

86 - 89 A-

81 - 85 B+

76 - 80 B

71 - 75 B-

67 - 70 C+

63 - 66 C

60 - 62 C-

57 - 59 D+

53 - 56 D

<53 F

*Note that although a ‘C’ represents a passing grade in any particular module, students are required to maintain a 2.5 average in their professional semesters.

**Note also that a ‘D’ is an unsatisfactory grade for your professional semester and will likely lead to your being asked to withdraw from the program.

Failure to meet a deadline without the prior consent of the instructor (based on medical or extenuating circumstances) will result in a lower grade for that assignment. Attendance is mandatory in all Professional Semester modules; unexcused absences may result in your being asked to withdraw from the program. Email the instructor (Runte@uleth.ca) or leave a phone message on office voice mail (403-329-2454) or with Margaret Beintema (403-329-2732), if you are going to be absent.

With the exception of the final examination and in-class workshops, students are required to word process all assignments.

Students are encouraged to use inclusive (e.g., non-sexist, non-racist) language in this course. Materials submitted for grading that fail to demonstrate inclusive language use may be docked marks.

[Please Note: All graded assignments must be picked up prior to Feb 15, 2010.]

Assignments

There are three assessments in this PSI module:

the key dimensions of what is to be measured, neither including irrelevant criteria nor excluding necessary elements; and that the criteria ‘make sense’ to the student (and their parents). Ensure that all the categories are defined sufficiently clearly that all observers using the rubric/checklist would interpret it in the same way (inter-rater reliability). Ensure that categories are clearly distinguishable and consistent between scales. Ensure that the checklist/rating scale/recording/-feedback sheet format is convenient and manageable; e.g., that you provided spaces for student name(s), class, and date, etc.

And so on. [These examples are only intended to illustrate the sorts of things the marker will be looking for, not an exhaustive checklist. You will be expected to meet all of the criteria (as set out in class discussion and the course text) appropriate for your chosen task.]

Submit your rubric along with the following report:

(A) Relevant background information about the class or group. (3 marks)

State the grade or age level for which you intend this rubric, and if it is subject-specific, state which subject.

(B) Purpose of the evaluation and outcomes to be evaluated. (4 marks)

State which objectives you have selected to evaluate and why. Explain how your assessment relates to the provincial curriculum, i.e., specify which goal from the Program of Studies you are assessing. (Note that you must refer to a specific objective from the provincial curriculum and not just a particular textbook or teacher-developed lesson plan.) State what information your rubric/checklist is to collect about those objectives and whether it is for diagnostic, placement, formative, summative, or program evaluation purposes.

(continued next page...)

Explain how the checklist you have designed or chosen matches the learning objectives you wish to measure (i.e., is a valid measure of your objective). Explain why the approach you have chosen is the most appropriate of the available alternatives. Explain how you decided what the appropriate standards would be for this grade/age level.

(D) Description of, and rationale for, the administrative procedures used. (4 marks)

Describe the procedures you would use to administer your rubric/checklist. (The plan you develop must be realistic and factor in the many other demands on a teacher’s time.) Explain how the procedures you intend to use would ensure that the data you gather is reliable.

Total this assignment: 35 Marks

Note: You may submit joint efforts for this assignment provided that everyone contributes equally to the effort and is prepared to accept the same grade on the joint submission. The scoring rubric for this assignment is based on the above descriptors; for the descriptors in rubric format, see Blackboard under ‘scoring rubrics’.

Alternative to Rubric Assignment:

Design a checklist/rating scale and rubric for the ‘In Depth’ Assignment for this class (see below). The requirements are the same as above, only focused on Ed 3504 objectives rather than K-12 content. See WebCT under ‘resources’ for details.

‘In Depth’ Evaluation Assignment Weighting = 35%

Write an ‘In Depth’ Evaluation Manual for Your Peers

This assignment is premised on the belief that the best way to learn something is to teach it to somebody else. Consequently your report may take the form of a how-to-evaluate-using-[fill in the blank]-manual, a set of Education 3504 lesson plans, an annotated Power Point presentation on an evaluation technique, a detailed Ed 3504 class handout, an audio podcast on an Ed 3504 concept, an instructional video on evaluation technique, a submission to www.videojug.com on how to evaluate or a web page related to this course. Students retain copyright on anything they produce, but it may help to focus your efforts if you think in terms of producing a useable resource for next years’ Ed 3504 class.

(continued next page...)

‘In Depth’ Evaluation Assignment (continued from previous page)

(A) Content (20 marks)

Explain how one would apply your chosen technique(s) to your major. For example, your mini-manual or power point presentation might be entitled ‘Using Higher Order Questions in the Grade 3 Social Studies Class’ or ‘Developing Portfolios for Elementary Art Class’ The point is to take one or more assessment techniques and provide in depth directions on how to implement these strategies in your grade and major.

The materials you develop must be self-contained and address all of the information you (or your peers or a classroom teacher) would need to successfully implement these techniques.

Provide a rationale for each instruction (i.e., explain the ‘why’ for each ‘how’). Identify any issues or potential pitfalls with your chosen technique and state your own position or strategy for mitigating these potential problems

Alternatively, you may explore particular evaluation concepts (validity, reliability, norm vs. criterion referenced grading, trends in Alberta evaluation, the problem of bias in student evaluation, etc.) rather than a particular assessment technique. Typical assignments might be a web page explaining validity and reliability, how they apply to classroom assessment, and why student teachers should care; or a comprehensive handout explaining the advantages and disadvantages of criterion vs. norm referencing, and when each is appropriate; or a detailed guide for student teachers on identifying and eliminating bias in one’s own evaluation.

Content will be graded for usefulness, completeness, accuracy, and thoughtfulness.

(B) Research and Citation (5 marks)

Although you are welcome to use information from class lectures and the course text in developing your own materials, these will not be sufficient to complete this assignment. The course can only provide an overview of available techniques; you must access additional resources to provide the greater depth required to make yourself an expert on this topic. You are welcome to use print or online materials, primary or secondary sources, but you must provide references for every idea cited, and include a list of references / a bibliography.

[Note that it is crucial not to plagiarize when developing instructional material; students caught plagiarizing may be assigned an ‘F’ in this course and so automatically withdrawn from the program. If you have questions about what constitutes plagiarism and how to avoid it, consult the UofL Library’s on-line Guide ‘Avoiding Plagiarism’ at http://home.uleth.ca/lib/guides/plagiarism.shtml]

Continued on next page ‘

class=Section13>

(‘ scoring rubric for ‘in depth’ assignment, continued from previous page)

In addition to content, your assignment will be evaluated for such factors as clarity, precision, and conciseness; in other words, how ‘user-friendly’ your materials are. Materials should also be engaging. Ask yourself, had these materials been created by someone else and assigned reading/viewing in my Ed 3504 class, how pleased would I be with them?

The precise criteria for the quality of the presentation will vary depending on which format you choose (e.g., lesson plan vs. manual vs. video, etc.), so you need to review the relevant scoring rubric for your specific chosen format on Blackboard before beginning your project. (If you choose a completely unique format, you may need to develop the rubric in conjunction with the instructor.)

(Note that you may choose to develop your own rubric for this assignment as your submission to the previous ‘rubric assignment’ in this course.)

Note: You may submit joint efforts for this assignment: that is, if two or more student teachers are teaching the same major at the same grade level or are interested in the same evaluation concepts, you may work together, provided that everyone contributes equally to the effort and is prepared to accept the same grade on the joint submission.

Alternative Assignment:

You may, if you are more comfortable with traditional research papers, investigate some topic related to this course, such as ‘The Origins, Assumptions, and Current Critique of Bloom’s Taxonomy’ or ‘The History of Assessment in Alberta Public Schools’. You are required to obtain the instructor’s approval in writing for research paper topics prior to beginning work to ensure your chosen topic is appropriate. (Failure to obtain written permission to undertake alternative assignment could result in a grade of zero.)