three colorful, overlapping circles representing various facets of holistic teaching evaluations

Holistic Teaching Evaluations

A comprehensive approach to measuring teaching effectiveness beyond student course evaluations.

Student surveys don't measure good teaching.

Research has consistently shown that student evaluations of teaching (SETs) are not valid instruments for measuring teaching quality. Modern approaches to teaching evaluation use evidence-based tools to promote fairness, encourage pedagogical creativity, focus on student learning over instructor popularity, serve both formative and summative purposes, and give all instructors the opportunity to put their best foot forward.
  • A 2021 meta-analysis of 90+ articles found that student course evaluations “are shaped by discipline, student interest, class level, class difficulty, class meeting time, and other course-specific characteristics, but not generally by actual instructor quality.”
  • A 2021 systematic analysis of 293 publications found that "SET results are strongly influenced by external factors unrelated to course content or teaching performance…based on the teaching academic’s gender, sexuality, ethnicity, age or disability as well as other marginalizing factors.”
  • Researchers have recently found that student evaluation scores show a stronger correlation with instructors' perceived “hotness” and “likeability” than with their teaching quality.
  • In 2020, a computational simulation found that “using SETs to identify poor teachers can result in an unacceptably high error rate even under the most optimistic scenarios supported by empirical research.”

Teaching can't be evaluated with a single metric.

The complexity of teaching requires a multi-faceted approach to evaluation that includes a variety of perspectives and evidence. The Teaching Analysis Tool triangulates three "lenses" of evaluation so that no one data point is used in isolation. Below are examples of forms of evidence that can paint a more complete picture of an instructor's teaching practice.
Three
Lenses
of Evaluation
three camera lenses: one each representing the instructor lens, the peer lens, and the student lens.
Instructor Lens
  • Teaching statement
  • Self-reflection or journal
  • Course materials and syllabi
  • CV or publications on teaching
Peer Lens
  • Classroom observation
  • Review of course materials
  • Review of student work or syllabi
  • Review of lesson plans or assessment designs
Student Lens
  • Interviews
  • Focus groups
  • Examples of student work
  • Surveys

Defining "good" teaching isn't guesswork.

Developing, recognizing, and rewarding effective teaching relies on clear, consistent, evidence-based criteria. While traditional teaching portfolios may assemble a hodgepodge of materials, they often lack a coherent framework that demonstrates how each piece fits into the bigger picture. The Teaching Analysis Tool evaluates evidence against seven dimensions of effective teaching—each distilled from decades of educational research. Institutions and departments can fully customize these dimensions to fit their unique context.
  1. Guiding Questions
    • What are students expected to learn from the courses taught?
    • Are learning goals clearly articulated to all students?
    • Are course goals appropriate for the course as part of the larger curriculum?
    • Are topics appropriately challenging and related to current issues in the field?
    • Are the materials high quality and aligned with course goals?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Course goals are appropriate for curriculum and are communicated to the students.
    • Content is at the appropriate level for the course.
    • Curriculum aligns with learning goals.
    • Content is innovative/related to current issues and developments in the field.
    • Course goals and materials are structured to support diverse learners.
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  2. Guiding Questions
    • How is in-class and out-of-class time used?
    • What assignments, assessments, and learning activities are implemented to help all students learn?
    • Are effective or high-impact methods being used to improve understanding and engage all students in learning?
    • Do in and out of class activities provide opportunities for practice and feedback on important skills and concepts?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Courses are well-planned and organized.
    • The instructor includes quality materials that are well-aligned with the learning goals.
    • Use of effective, evidence-based methods to enhance student understanding.
    • Students have opportunities to practice skills embedded in course learning goals.
    • Assessments/assignments are appropriately challenging, tied to course goals, and provide differentiation for diverse learners.
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  3. Guiding Questions
    • What impact do these courses have on learners?
    • What evidence shows the level of student understanding?
    • Are approaches to evaluating students equitable?
    • Are standards for evaluating students connected to program or other expectations?
    • Are there efforts to support learning in all students and reduce inequities?
    • Does learning support success in other contexts (e.g., later courses, laboratories, field work, practica)?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Standards for evaluating student understanding are clear and align with course goals and content.
    • Student learning meets department expectations (discuss criteria and how to assess).
    • Some use of evidence of student learning to inform teaching.
    • Quality of learning is likely to promote success in other courses/contexts.
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  4. Guiding Questions
    • Is the classroom climate respectful and cooperative?
    • Does it encourage motivation and engagement for all students?
    • Do all students feel included?
    • What are the students’ views of their learning experience? How has student feedback informed the faculty member’s teaching?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Classroom culture is inclusive and promotes engagement and respect.
    • Classroom culture appears to broadly encourage student participation.
    • No consistently negative student ratings of teacher accessibility or interaction skills.
    • Instructor articulates some lessons learned through student feedback.
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  5. Guiding Questions
    • How effectively has the instructor worked individually with undergraduate or graduate students?
    • Is the individual work of the instructor with students equitable and responsive to diverse students?
    • How does the quality and time commitment to mentoring fit with disciplinary and departmental expectations?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Some evidence of effective advising and mentoring (define as appropriate for discipline).
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  6. Guiding Questions
    • How has the instructor’s teaching changed over time?
    • How has this been informed by evidence of student learning and student feedback?
    • Have improvements in student learning been shown, based on past course modifications?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Continued competent teaching, may include reflection based on input from peers and/or students.
    • Articulates some lessons learned or changes informed by prior teaching, and student learning. May include reflection on student work.
    ADVANCED
    The instructor surpasses the criteria for Proficient.
  7. Guiding Questions
    • In what ways has the instructor contributed to the broader teaching community, both on and off campus?
    • Is the instructor involved in teaching-related committees, curriculum or assessment activities?
    • Does the instructor share practices or teaching results with colleagues?
    Sample Rubric
    DEVELOPING
    The instructor does not yet meet the criteria for Proficient.
    PROFICIENT
    • Has made some contributions to teaching and learning culture in department or institution.
    • Some engagement with peers on teaching.
    • Has shared teaching practices or results with others (e.g., presentation, workshop, essay).
    ADVANCED
    The instructor surpasses the criteria for Proficient.

Legacy evaluation systems are ripe for change.

Over the past 8 years, the Transforming the Evaluation of Teaching (TEval) Project has leveraged a $2 million National Science Foundation grant to coalesce a nationwide movement around holistic teaching evaluations. Our established 4-step method for transitioning to a holistic approach—which has already impacted hundreds of instructors across multiple campuses—makes it easy to start using the Teaching Analysis Tool in your institution or department.
  1. 1
    Discuss and adapt the dimensions of teaching
    • Engage faculty in discussion around evaluation.
    • Reach broad consensus on working descriptions of each dimension.
  2. 2
    Agree on criteria for proficiency
    • Discuss department guidelines for proficiency for each of the seven dimensions.
  3. 3
    Carry out small pilots
    • Select groups of pilot participants.
    • Decide on evidence and dimensions to pilot.
    • Engage department in discussion of pilot findings.
  4. 4
    First implementation
    • Decide whom to include and which dimensions will be evaluated.
    • Develop a schedule for future evaluations.

Technology streamlines teaching reviews and analyses.

Robust evaluation workflows involve multiple users, supporting documents, evaluation rubrics, and timelines. Relying on generic cloud storage platforms like Google Drive, Microsoft OneDrive, and Interfolio makes data hard to analyze and overwhelms faculty with logistical busywork. The Teaching Analysis Tool is a custom-built platform designed by college teachers and administrators to analyze teaching effectiveness and generate comprehensive reports compatible with existing Review, Promotion, and Tenure systems.
Example Report Summary
Teaching Dimension
Instructor Lens
Peer Lens
Student Lens
Proficiency
1. Goals, Content, and Alignment Grading rubrics Classroom observation Proficient
2. Teaching Practices Lesson plan Student surveys Proficient
3. Student Progress Toward Learning Goals Review of student work First/final drafts Advanced
4. Classroom Culture and Student Perceptions Syllabus Formative assessment Proficient
5. Mentoring and Advising Teaching statement Student letters Proficient
6. Reflection and Iterative Growth Reflection journal Teaching center letter Advanced
7. Teaching Service, Scholarship, or Community CV Collaborator letter Developing

Three ways to get started!

It's hard to value or reward good teaching until you have reliable ways of measuring it. And it's hard to measure good teaching unless you have the right tools at your disposal. What will your next step be towards more holistic teaching evaluations? Here are three great places to start: