Contents
Goals
Topics
Schedule
Texts
Assignments
Moodle
UofL

Evaluation of Learning
Ed 3504 Fall 2011 Sections GHI & PQR

Instructor: Robert Runté, PhD                    Office: TH313                   Phone: 403-329-2454

E-mail: Runte@uleth.ca Please Note: all emails to instructor must include "Ed3504 GHI" (or PQR) in subject line to avoid automatic deletion by anti-spam software. Emails are read during office hours and answered within 3 working days.

Instructor website:     http://www.uleth.ca/edu/runte/           Instructor Assessment Blog

Office Hours: I am generally in my office when not in class or meetings, and I have an open door policy: if you see me in my office, you are invited to drop in with your questions. My meeting schedule this term, however, is somewhat erratic, so if you want to be sure to catch me in, you must make an appointment. Talking to me at class break, email, or phone are the best ways to make an appointment to see me. Please note that I schedule my own appointments, and that support staff do not have access to my schedule.

Administrative Support:        Margaret Beintema        Phone 403-329-2732        Office: TH321

PQR     Tuesdays       1:00 — 3:50           Room TH277
GHI     Wednesdays   9:00 — 11:50         Room TH373

Course Description

Calendar Description: Introduction to a variety of approaches to evaluating student learning.

This is the first of two evaluation courses you will take as part of your professional training. Both courses emphasize the practical skills and knowledge you will require to become a successful classroom teacher.

This course provides a general introduction to evaluation issues, terminology, and principles. We will also be exploring a number of non-test assessment techniques, such as:

  • performance-based assessment
  • observation checklists
  • rubrics
  • anecdotal records
  • learning logs and journals
  • portfolios
  • student and peer evaluation
  • self-evaluation
Relationship of Education 3504 to the Practicum:

The practicum in this first professional semester requires student teachers to take responsibility for individual lessons. Consequently, discussion in this course is focused on how teachers can incorporate evaluation into their daily lesson planning to ensure that their instruction is effective. In other words, it is designed to help you answer the question, "Did the students learn what I was trying to teach them today?"

Discussion of evaluation techniques such as the multiple-choice questions and essay tests will be left until Professional Semester II when student teachers will be taking responsibility for teaching entire units in their practicum, including the construction of unit tests and essay assignments.

Relationship of Education 3504 to Provincially Mandated KSAs:

The Minister of Education has established a list of knowledge, skills, and attributes (KSAs) required for Interim Certification as a classroom teacher in Alberta. Graduates may be asked to document that they possess these KSAs. The KSA applicable to this module of Professional Semester I is listed below, and can be found with the entire list of KSAs, at: http://ednet.edc.gov.ab.ca/educationguide/pol-plan/polregs/421.asp

    (K) the purposes of student assessment. They know how to assess the range of learning objectives by selecting and developing a variety of classroom and large-scale assessment techniques and instruments. They know how to analyse the results of classroom and large scale assessment instruments including provincial assessment instruments, and how to use the results for the ultimate benefit of students.

Course Goals
By the end of the course, students will be able to:

  • speak knowledgeably about trends and issues in evaluation, such as the on-going debate over standardized tests, and the differences between assessment of learning, assessment for learning, and assessment as learning
  • understand and appreciate the role and importance of evaluation in monitoring one's own teaching effectiveness
  • understand the difference between formative, summative, placement, and diagnostic evaluation, and use appropriate evaluation strategies in each context
  • define criterion-referenced and norm referenced evaluation, and interpret student results in the appropriate context
  • define and distinguish between validity and reliability
  • know and apply Bloom's Taxonomy (revised version) of the Cognitive Domain
  • design learning outcomes for individual lessons (e.g., write learning objectives)
  • relate objectives and evaluation to the provincial curriculum
  • identify potential sources of error and bias in alternative assessments, and how to address these
  • list the advantages and disadvantages of each (non-test) evaluation strategy
  • recall and apply basic principles of observational techniques
  • recall and apply basic principles of performance based assessment
  • recall and apply appropriate techniques in recording anecdotal records (using episode, performance, & time sampling)
  • design and use a checklist and rubric
  • identify and apply appropriate rating scales
  • design and manage learning logs, blogs, and journals
  • design appropriate assessment strategies using portfolios
  • identify appropriate strategies for a teacher/parent/student conference
  • demonstrate appropriate oral questioning techniques in the practicum
  • incorporate evaluation in their daily lesson planning and instruction in the practicum

Back to top

Topic Outline

  1. . Evaluation Issues
    1. The Who, What, When, and Why of Evaluation
    2. Trends and Issues in Evaluation: Accountability, Standardized Testing, and Alternative / Authentic Assessment
    3. Assessment of learning vs. Assessment for learning vs. Assessment as Learning
  2. The Language of Evaluation
    1. Basic Terms
      1. Use: Placement, Formative, Diagnostic, Summative
      2. Interpretation: Criterion vs. Norm reference
    2. Bloom's Taxonomy (revised version)
  3. Curriculum and Evaluation
    1. Alberta Curriculum and Evaluation
    2. The Diagnostic Teaching Model
    3. Writing Specific Learning Objectives
  4. Criteria for Effective Evaluation
    1. Basic Principles of Systematic Evaluation
    2. Introduction to Validity and Reliability
    3. Ethical Issues in Evaluation: Principles For Fair Student Assessment Practices For Education In Canada
  5. Evaluation Techniques (The "How To" of Evaluation)
    1. Oral Questioning and Class Discussion as Evaluation
    2. Deskwork
    3. Authentic Assessment
      1. Basic Principles
      2. Performance Based Assessment
    4. Observational Techniques
      1. Basic Principles
      2. Anecdotal Records (Episode, Performance, & Time Sampling)
      3. Checklists
      4. Rating Scales
      5. Worthy Tasks Rubrics
      6. Potential Sources of Error
    5. . Learning Logs and Journals
    6. Portfolios
  6. The Teacher/Parent/Student Conference

Back to top

 


Tentative Schedule

Class Date Topics Readings (5th Edition) *
Class 1 September
13 PQR
14 GHI
  • Intro: The Who, What, When & Why of Evaluation
  • Trends in Evaluation
  • Assessment of learning vs. Assessment for learning vs. Assessment as Learning
 Intro (pp.1-12)
Class 2 September
20 PQR
21 GHI
  • Placement, Formative, Diagnostic, & Summative evaluation
  • Criterion vs. Norm reference
  • 3 Domains & Bloom's Taxonomy (revised)
  • validity & reliability
  • Learning Objectives
  • role & importance of evaluation in monitoring one's own teaching effectiveness
Chapter 1 (pp. 13-27)
Class 3 September
27 PQR
28 GHI
  • Observation
  • Overview of Authentic Assessment

[Proposal due: Declare topic & mode for 'Peer Teaching' assignment] 		 
Class 4 October
4 PQR
5 GHI
  • Performance Based Assessment
  • Checklists
  • Rubrics
 Chapter 4 (58-78)
Chapter 5 (79-106)
Class 5 October
11 PQR
12 GHI
  • Oral Questioning (& metacognition)

[Checklist/Rubric assignment due]		  
Class 6 October
18 PQR
19 GHI
  • Portfolios/conferencing
  • Journals and learning logs
 Chapter 3 (40-57)
Chapter 9 (157-168)
Class 7 October
25 PQR
26 GHI
  • Principles of Fair & Accurate Assessment
[Peer Teaching Assignment Due] 		Principles document (on-line download)
Class 8 [Exam] November
1 PQR
2 GHI		 [Course Examination] all the above, plus
Chapter 2 (pp.28-39)
Chapter 6 (pp107-128)
Chapter 7 (pp129-139)

*Note: You will not require Chapter 8 of the text until PSII evaluation module.

Back to top

Required Texts

 

    Kay Burke, How To Assess Authentic Learning, 5th Edition and is available in the UofL Bookstore.

    Earlier editions of this book are also acceptable, so feel free to buy used. (Older covers shown below left.)

    You may obtain a copy of Principles for Fair Student Assessment Practices for Education in Canada free, from www.bced.gov.bc.ca/classroom_assessment/fairstudent.pdf and/or on Moodle.

    You may also find it useful to access the AAC site at http://www.aac.ab.ca/ user name: uleth

    You may also wish to consult the UofL Library's on-line Guide, "Avoiding Plagiarism" http://home.uleth.ca/lib/guides/plagiarism.shtml


Back to top

Assignments

There are four formal assessments (and five deadlines) in this PSI module:
Class-designed assignment (10%) due Sept TBA (set by class vote)
"Peer Teaching" Proposal deadline (0%) due PQR Sept 27 GHI Sept 28
Checklist/Rubric Construction (30%) due PQR Oct 11 GHI Oct 12
"Peer Teaching" Assignment (30%) due PQR Oct 25 GHI Oct 26
Course Examination (30%) PQR Nov 01 GHI Nov 02
To reduce paper consumption (consistent with UofL policies on sustainability) and to facilitate the logistics of marking, wherever possible, please submit all assignments via Moodle. When submitting group assignments, one person submits the assignment (with all the group members' names on it), and the others submit a note stating whose name the assignment is under. (This step of submitting a one-line note is necessary for Moodle to unlock your grade in the grade sheet and makes life a great deal easier for the marker.) If you encounter difficulties uploading an assignment to Moodle, you can always email it directly to instructor (Runte@uleth.ca)..

Class-Designed Assignment                 Weighting = 10%

The class will work together to design an appropriate formal evaluation of course content worth 10% of the course. The exact due date will be established by the class, but will be in September. The class will design the rubric/scoring criteria.

Checklist/Rubric Assignment               Weighting = 30%

Design a rubric, and an associated checklist/rating scale, to assess some aspect of the Alberta K-12 curriculum.

Scoring Criteria
The rubric and checklist free of technical flaws

Be as concise and precise as possible. Avoid generalities and focus on specifics. Ensure that your rubric has face validity: i.e., adequately addresses the key dimensions of what is to be measured, neither including irrelevant criteria nor excluding necessary elements; and that the criteria 'make sense' to the student (and their parents). Ensure that all the categories are defined sufficiently clearly that all observers using the rubric/checklist would interpret it in the same way (inter-rater reliability). Ensure that categories are clearly distinguishable and consistent between scales. Ensure that scoring categories are accurately but tactfully labeled. Ensure that the checklist/rating scale/recording/feedback sheet format is convenient and manageable; e.g., that you provided spaces for student name(s), class, and date, etc.

And so on. [These examples are only intended to illustrate the sorts of things the marker will be looking for, not an exhaustive checklist. You will be expected to meet all of the criteria (as set out in class discussion and the course text) appropriate for your chosen task.]

Submit your rubric and checklist along with the following report:

(15 marks)
(A) Relevant background information about the class or group.

State the grade or age level for which you intend this rubric, and if it is subject-specific, state which subject.

(3 marks)
 

(B) Purpose of the evaluation and outcomes to be evaluated

State which objectives you have selected to evaluate and why. Explain how your assessment relates to the provincial curriculum, i.e., specify which goal from the Program of Studies you are assessing. (Note that you must refer to a specific objective from the provincial curriculum (citing objective number or page number) and not just a particular textbook or teacher-developed lesson plan.) State what information your rubric/checklist is to collect about those objectives and whether it is for diagnostic, placement, formative, summative, or program evaluation purposes.

(4 marks)
&bsp;

(C) Description of, and rationale for, the evaluation procedures chosen.

Explain how the checklist you have designed or chosen matches the learning objectives you wish to measure (i.e., is a valid measure of your objective). Explain why the approach you have chosen is the most appropriate of the available alternatives. Explain how you decided what the appropriate standards would be for this grade/age level.

(4 marks)
 
(D) Description of, and rationale for, the administrative procedures used

Describe the procedures you would use to administer your rubric/checklist. (The plan you develop must be realistic and factor in the many other demands on a teacher's time.) Explain how the procedures you intend to use would ensure that the data you gather is reliable.

(4 marks)
Total this assignment: 30 Marks

Note: You may submit joint efforts for this assignment provided that everyone contributes equally to the effort and is prepared to accept the same grade on the joint submission.

Note: the scoring rubric for this assignment is based on the above descriptors; for the descriptors in rubric format, see Moodle under "Resources".

Questions about this assignment? See the Frequently Asked Questions File on Moodle under "Resources".

Alternative to Rubric Assignment:

Design a checklist/rating scale and rubric for the "Peer Teaching" Assignment for this class (see below). The requirements are the same as above, only focused on Ed 3504 objectives rather than K-12 content. See Moodle under "resources" for details.

Back to top

 

Peer Teaching Assignment               Weighting = 30%

This course introduces you to a variety of assessment techniques, concepts, and issues, and consequently cannot go into as much detail on any of them as one might wish. The purpose of this assignment, then, is to allow you to become expert on at least one classroom evaluation technique, concept, or issue. Choose one or more evaluation technique(s), concept(s) or issues reltated to this course you wish to learn more about and....

Provide a learning object to your peers on some aspect of this course module.

This assignment is premised on the belief that the best way to learn something is to teach it to somebody else. Consequently, instead of writing yet another term paper for your instructor (who presumbably already knows something about these topics), you are going to prepare learning resources for your classmates.

Your report may take the form of a 'how to' manual, an annotated Power Point presentation, an annotated Smartboard Notebook lesson, a set of Education 3504 lesson plans, a detailed class handout, an audio podcast, an instructional video, a submission to www.videojug.com, a web page, a wikki, a Facebook group, or any other format that you think will engage your peers.

Your assignments will then be posted to the class Moodle site for your peers to share.

Note that this is not a class presentation in the usual sense — this is not an assignment on micro teaching or group presentation. A secondary purpose of this assignment is to have you start thinking about "teaching" as more than just lecturing and class discussion. Preparing curricular materials that engage and inform students is also an important aspect of successful classroom teaching. Integrating technology, creating 'learning objects' and preparing notes and handouts are all part of your new career.

Possible Topics:

  • Evaluation Technique: Choose an evaluation technique covered in this module (oral questioning, anecdotal records, performance assessment, portfolios, reliability/validity, norm/criterion referencing, conferencing, learning logs, journals, peer /self-evaluation, etc.) and explain how one would apply your chosen technique(s) to your major/grade level. For example, your mini-manual or power point presentation might be entitled "Using Higher Order Questions in the Grade 3 Social Studies Class" or "Developing Portfolios for Elementary Art Class" The point is to take one or more assessment techniques and provide in depth directions on how to implement these strategies in your grade / major.

  • Evaluation Concepts: Develop materials to teach evaluation concepts such as validity / reliability, norm vs. criterion referenced grading, etc., rather than a particular assessment technique. Typical assignments might be a web page explaining validity and reliability, how they apply to classroom assessment, and why student teachers should care; or a comprehensive handout explaining the advantages and disadvantages of criterion vs. norm referencing, and when each is appropriate.

  • Bias: Identify the problem of bias in student evaluation, looking at such issues as gender differences, racism, social class, religious differences or cultural misunerstanding in student evaluation. Or develop a detailed guide for student teachers on how to identify and eliminat bias in one's own evaluation practices.

  • Differentiated Instruction / Evaluation: Investigate how to adapt your assessment practices to accommodate various special needs — ESL, immigrant cultures, physical, mental or behavioral conditions, the gifted, or some other population.

  • FNMI (First Nations, Meti, and Inuit): How do our current assessment practices discriminate against FNMI students? How can our assessment practices be adapted to respect and incorporate FNMI cultures? (Note that we are not concerned only with assignment content here, but primarily with processes.)

  • 21st Century Learning and Learners: Examine the implications from this important Alberta Education document for the topic of evaluation

  • Another Course-related Topic. You may ask the instructor for permission to come up with your own relevant topic. You must have your topic and format approved in writing by the instructor before proceeding, or risk getting a zero on the assignment if it is subsequently deemed off topic. But if it is a topic of interest to you and seems to relate to student evaluation, what can you lose by asking if you can do it?

Scoring Criteria
(A) 'Learning Object' Content

The materials you develop must be self-contained and address all of the information you (or your peers or a classroom teacher) would need to successfully implement the evaluation techniques you have chosen; or to understand the concept; or to be able to intelligently discuss the issues you have identified.

For example, if you are focusing on an evaluation technique, you will need to provide a rationale for each instruction (i.e., explain the "why" for each "how") and to identify any issues or potential pitfalls with your chosen technique; and state your own position or strategy for mitigating these potential problems.

Similarly, if you are discussing an issue in evaluation, you must explain all sides of the issue (an issue always has at least two sides — otherwise, it's not an issue) and make recommendations for how you would attempt to resolve it for Alberta schools or in your own classroom. If you are addressing an issue or trend, you should take a position and defend it.

Content will be graded for usefulness, completeness, accuracy, and thoughtfulness.

(Of course, the evaluation technique, concept, or issue must be one of those listed under course goals or topics for Ed 3504, or one for which you have been given written permission by the instructor — see above.)

(15 marks)
(B) Research and Citation

Although you are welcome to use information from class lectures and the course text in developing your own materials, these will not be sufficient to complete this assignment. The course can only provide a superficial overview of available techniques, concepts, trends and issues; you must access additional resources to provide the greater depth required to make yourself an expert on this topic. You are welcome to use print or online materials, primary or secondary sources, but you must provide references for every idea cited, and include a list of references / a bibliography.

[Note that it is crucial not to plagiarize when developing instructional material; students caught plagiarizing may be assigned an "F" in this course and so automatically withdrawn from the program. If you have questions about what constitutes plagiarism and how to avoid it, consult the UofL Library's on-line Guide "Avoiding Plagiarism" at http://home.uleth.ca/lib/guides/plagiarism.shtml.]

(5 marks)
(C) Quality of Presentation

In addition to content, your assignment will be evaluated for such factors as clarity, precision, and conciseness; in other words, how 'user-friendly' your materials are. Materials should also be engaging. Ask yourself, had these materials been created by someone else and assigned reading/viewing in Ed 3504 class, how pleased would you be with them?

The precise criteria for the quality of the presentation will vary depending on which format you choose (e.g., lesson plan vs. manual vs. video vs. powerpoint etc.), so you need to obtain from your instructor a copy of the specific scoring rubric for your chosen format before beginning your project. (If you choose a completely unique format, you may need to develop the rubric in conjunction with the instructor.)

(10 marks)
Total this assignment: 30 Marks

(Note that you may choose to develop your own rubric for this assignment as your submission to the previous "rubric assignment" in this course.)

Note: You may submit joint efforts for this assignment: for example, if two or more student teachers are teaching the same major at the same grade level or are interested in the same evaluation concepts or issues, you may work together, provided that everyone contributes equally to the effort and is prepared to accept the same grade on the joint submission.

Questions about this assignment? See the Frequently Asked Questions File on Moodle under "Resources".


Course Examination Weighting = 30%

The course examination is a closed-book test with true/false, matching, multiple-choice and open-ended questions. It will focus primarily on material covered in class, but does include some questions based on material covered only in the texts. The best way to prepare for the examination is to attend class and read the assigned readings.

Back to top

Grading

Excellent Good Satisfactory* Poor** Failing
96 - 100     A+
90 - 95       A
86 - 89       A-		 81 - 85       B+
76 - 80       B
71 - 75       B-		 67 - 70       C+
63 - 66       C
60 - 62       C-		 57 - 59      
D+
53 - 56       D 		less than 53 = F

*Note that although a "C" represents a passing grade in any particular module, students are required to maintain a 2.5 average in their professional semesters.

**Note also that a "D" is an unsatisfactory grade for your professional semester and will likely lead to your being asked to withdraw from the program. (I have only ever given out one 'D' in 21 years of teaching this module.)

Failure to meet a deadline without the prior consent of the instructor (based on medical or extenuating circumstances) will result in a lower grade for that assignment.

Attendance is mandatory in all Professional Semester modules; unexcused absences may result in your being asked to withdraw from the program. Email the instructor (Runte@uleth.ca) or leave a phone message with Margaret Beintema (403-329-2732), if you are going to be absent.

With the exception of the final examination and in-class workshops, students are required to word process all assignments. Students are encouraged to use inclusive (e.g., non-sexist, non-racist) language in this course. Materials submitted for grading that fail to demonstrate inclusive language use may be docked marks.

[Please Note: All graded assignments must be picked up prior to Feb 15, 2012.]

Back to top

Cell phone policy: You are generally asked to turn your cell phone off during class. There may be occasions when you need to be available for a call during class time (e.g., issues related to childcare, employment, etc.): To minimize the disruption of such calls to the rest of the class, please give the instructor advance warning that you are expecting or need to be available for such a call; set your phone to vibrate; and sit where you may exit the class discretely to take the call. The same guidelines apply to texting.

Laptop policy: You are welcome to bring your laptop to class. You are expected, however, to remain focused on the current class activity: using your laptop for unrelated tasks is distracting and discourteous to the presenter and to your classmates. (Please remember that, unlike television, the presenter can see you.) It is your responsibility to remain fully engaged in, and contributing to, the success of the class.

Back to top

Last updated August 30, 2011.