Using Classroom Observations to Measure Teacher Effectiveness
- Articles, Blog

Using Classroom Observations to Measure Teacher Effectiveness


ELIZABETH GRENINGER:
Good afternoon, everyone. Thank you for joining us in the
seventh webinar for the Teacher Effectiveness Series. Today’s webinar is on Classroom
Observations to Measure Teacher Effectiveness, and it’s
being hosted by the Regional Educational Laboratory
of the Mid-Atlantic. In order to minimize disruptions
throughout the webinar, we’ll be muting the line. I’m Elizabeth Greninger and I’ll
be facilitating today’s webinar. Our featured presenters today
are Rob Ramsdell and Christy McInnis from
Cambridge Education. This webinar series is a forum
for us to provide stakeholders in the Mid-Atlantic region
with opportunities to discuss multiple topics within the
field of teacher effectiveness. This is our sixth webinar, and
many of you I see have been attending some of
our prior webinars, so you’re understanding that
the goal of this series is to increase our participants’
understanding and use of the research-based teacher
effectiveness strategies. So, now we’d like to take
a moment to find out who is joining us on the webinar today. In a moment, you’ll see a pop-up
box come onto your screen with a multiple choice question. Click on the answer choice
that best indicates your professional role. Looks like we have a pretty
diverse audience today with quite a few
researchers joining us, and a good number of teachers
and other school district staff. We know that the content of
today’s presentation will really speak to all people who are
working in any capacity within school districts and states. Now I’d like to
introduce our presenters. We have with us today
Mr. Rob Ramsdell from Cambridge Education. He serves as the Vice President
of School Improvement Services at Cambridge Education. He also leads the Tripod Project
for School Improvement as a leading expert on educator
evaluation with its founder, Ron Ferguson. Mr. Ramsdell started his career
as a high school social studies teacher, and founded
FreshPond Education in 1996, which then joined Cambridge
Education in 2007. Mr. Ramsdell also served
as Senior Director of PBS TeacherLine, an online
professional development initiative funded by the U.S. Department of Education. He has experience working
with hundreds of schools and districts to design and
implement high quality school improvement programs
and professional development initiatives. Also joining us today as a
co-presenter is Christy McInnis. Christy serves as a Senior
Education Consultant for Cambridge Education. In her work, she develops
and leads school improvement efforts, primarily focused on
improving school leadership, teacher instruction,
and student learning, using teacher and principal
evaluation as the vehicle for assisting districts and
identifying areas of needed improvement. Ms. McInnis began her career in
1989 as an English teacher in Okaloosa County, Florida. After 14 years of
teaching grades 6-10, she assumed a school leadership
position allowing her to facilitate improvements in
curriculum and instruction in one of Florida’s top
ranked high schools. Prior to joining Cambridge, she
served as Okaloosa County School District’s Director of
Professional Development, Teacher and
Principal Evaluation, New Teacher Induction,
and Certification. At this time, I’d like to pass
the presentation along to Rob. ROB RAMSDELL: Great. Thanks a lot, Elizabeth. And it’s a privilege
to be here today. And I just want to say a moment
more about my experience and what’s brought me into
this particular work with educator evaluation. As Elizabeth’s
introduction highlighted, my work is mainly focused on
the Tripod Student Survey, working closely
with Ron Ferguson, who I think many of you
heard from on a webinar earlier in the series. But through that work and
our involvement with the MET (Measures of Effective
Teaching) Project, we’ve had the opportunity to get
heavily involved in work related to effective classroom
observation as well, which has also been a specialty
of Cambridge Education both here in the U.S., but also work
internationally where we support school improvement and work
related to educator evaluation across about 45 countries. So, it’s really my role here
today to kind of set some context for the webinar, and
quickly get it over to Christy, who has much more recent and
deeper experience with classroom observation and will be able
to share insights both from her experience with Okaloosa, but
also the work she’s doing out in the field in a number of states
supporting districts around this particular issue. For today’s session, I just want
to emphasize – and I want to highlight – we received
questions from people who shared those with us during the
registration process, and the range of questions was
significant and we’re hopeful that we’ll address many of
the issues that were raised in those questions. But I think we’re assuming,
going into this conversation, that we have an audience that
has a lot of understanding about issues related to classroom
observation already. And we’re hoping this webinar
provides an opportunity to build on what is likely already a
significant understanding by giving us some time to focus
in on this particular issue, to look at some of the research
implications involved with this work, to dig a bit into
the issue of feedback, so not just the
process of observation, but what happens after
observations to support professional learning. And also, through the kind of
polling and interactive work we’ll do throughout
the presentation, give folks an opportunity to
reflect on what you’re seeing in your local districts. Now, with that, thanks
to the initial poll, I want to acknowledge that we
have a lot of researchers who may not have – I think, about 50
percent of the participants here are researchers who may
or may not be working directly in a district. And what I’d ask you all to do
is try to situate yourselves in a district that you’re
most familiar with, because in a number of the polls
that we’ll be doing during the session, we’ll be asking you to
reflect on what’s happening in your district. And if you’re in a district
supporting a range of districts, then we’ll want you to try to
situate yourself in a district you’re most familiar with and
respond based on that particular experience and perspective. The session’s going to be
broken up into four sections. We’re in the middle here of
the introduction where I’ll be trying to set the context
for our time together. Christy will then dig into some
of the essentials of effective classroom observation. We’ll then focus a little bit on
the issue of reliability and the benefits of multiple observers. And then Christy will come back
in and focus on issues related to quality feedback and making
the link to high quality professional learning as part
of the observation process. So, I think most people on this
call are probably here in part because you’re in environments
where there is a shift of some sort occurring – a shift to
focus differently on classroom observation and to develop the
work of classroom observation particularly in relating it
to new teacher evaluation and educator evaluation systems. Now, we’re all also probably
aware that the quality of teaching is the single most
important factor in student learning, and this is what’s
driving a lot of the emphasis on changing and improving
teacher evaluation systems, such that they are rigorous,
valid, and reliable, and trusted by all stakeholders. But as we’re thinking
about this work, and particularly the focus
on classroom observation, but more broadly the work that’s
happening across the country around new teacher
evaluation systems, we need to acknowledge that work
isn’t occurring in a vacuum. And what I’m going to do here
is just quickly build with the outcome ultimately that we’re
all focused on at the top here with student achievement, but as
measured by a range of different types of achievement. So not just measured by
tests, but also attitudes and behaviors, educational
attainment, employment, and citizenship. And I think probably most all of
us on the call today would agree that the way we get there is
not just by improving educator evaluation systems and
classroom observations, but rather by building strong
district leadership and organizational capacity,
supporting strong school leadership and
organizational capacity, promoting effective learning
environments and engaged students, with the involvement
of families and communities. In the conversation
today about observation, we’re particularly focused on
the issue of effective student learning environments
and engaged students. But up front, I want to
acknowledge that this work of classroom observation and the
focus on effective student learning environments and
student engagement is occurring in this wider context. So, we’re all aware that teacher
evaluation systems are changing across the country
in a range of ways, whether it’s adding multiple
measures or advancements in important work like we see in
the advances in the Framework for Teaching rubric of
Charlotte Danielson, to the different forms of
consequences that are tied to educator evaluation and the role
of classroom observation in that observation process. And we’re going to quickly do a
poll just to try to gauge what kind of changes you’re
experiencing in the districts and settings that
you’re working in. And this, again, is where I’d
ask the researchers and other stakeholders in the audience
to try to situate yourself in a district that you’re most
familiar with when responding to this polling question that’s
going to appear momentarily. ELIZABETH GRENINGER: Okay,
so like Rob just mentioned, we’d like you to
respond to the question. What changes in teacher
evaluation are taking place in your school or district? And you can click more than one
response if multiple responses apply to your situation. Okay. So, there’s quite a range of
things happening in teacher evaluation across the
schools and districts we have represented. It looks like quite a few people
are using evaluation criteria and their student
standardized test scores – a good number of us. Looks like fewer people are
including incentives for strong performance on
evaluation at this point. Rob, did you have any thoughts
on what you see here in the responses from the audience? ROB RAMSDELL: No, it looks as
though we had a pretty good list and that these are the things
that folks are experiencing, and just to acknowledge that
it’s a wide range of issues. I’d like to maybe highlight
the bottom item – there are supports/interventions/trainings
available for teachers who perform poorly on evaluations. Not that it’s low, but it is
one of the lower areas of the options selected, perhaps
that was because it was at the bottom. But as we go through today’s
presentation and try to really focus on the step of making sure
that evaluation and classroom observation support growth
and professional learning, hopefully we’ll come back to
this issue of what kinds of supports are in place to follow
up and support what we’re learning through the
observation process. I think we can
hide the poll now. So, certainly, one of the things
that districts and states across the country are focused on in
this shift is the emphasis on multiple measures. And I think, again, a number
of you were participants in the webinar where Ron Ferguson
shared some of the experience with the Tripod Surveys. Of course, we have value-added
and growth data in the mix. And today, with our focus
on classroom observations. But with all of this, we just
want to continue to keep the focus on the outcome that we’re
aiming for in implementing educator evaluation systems,
which is more engaged students and higher achieving students
across the range of achievement outcomes that we all care about. I think everyone’s
probably aware of why this shift is happening. There’s a – certainly a
recognition that past evaluation systems have lacked rigor,
both in their thoroughness and accuracy, and that has
undermined the credibility of the evaluation systems. The quality of feedback
available to educators and teachers in particular
has been limited. And there are a range of policy
issues that have fueled a lot of the changes that we’re
experiencing across the country. Now, this next slide sort of
presents a theory of change here with the notion that better
classroom observations will provide more rigorous
evaluations that have higher levels of
thoroughness and accuracy, building collegial feedback
and professional learning. And I’ll just pause for a
moment and ask you to reflect on whether all three of these areas
are receiving equal attention in the settings where
you’re working. Certainly our experience
is, in many settings, there’s this very, very strong
emphasis on the evaluation system, but there has not yet
been as much of a focus on the next step of building the
kind of collegial feedback and professional learning that will
actually fuel improvement and growth in our classrooms
and in our schools. And that will be a theme that
we’ll come back to during the presentation today. So, as we think about this work
and move towards transitioning to Christy digging into more
details on the effective classroom observation work
that she’s been involved with, and others are involved with, I
think what we’re aiming for here is a system in classroom
observation approaches, in particular, that move us from
an emphasis on evaluation to a system that finds greater
balance and provides a picture that looks more like this. And with that, I’m going to
hand it over to my colleague, Christy, to dig in a bit more
on the effective classroom observation work that
she’s been involved with. CHRISTY MCINNIS: Thank
you, Rob, very much. Just kind of hinging on what
Rob shared just a moment ago, as we’re thinking about
observing teaching and evaluating teaching, we start
to move into our quest to just continuously analyze
student learning. So, as we start thinking
about the effective classroom observation and what the
essentials are for that process, I like to sometimes think of
those essentials as being sort of a checklist. And although we know that
teaching is far more complex than a checklist, there’s just
this kind of philosophy I have in thinking about an analogy,
like pilots in the cockpit of an airplane. Prior to take off, they use a
checklist to be sure that all of the important steps and
procedures have been completed before they dare take
off on that runway. So, they want to make sure that
there’s a successful flight, and they do that
with that checklist. So, similarly, there are
procedures that we can follow when using classroom
observations to ensure that we have more successful teacher
instruction and student learning taking place in the classroom. So, many of the things you see
here on this list are going to certainly be some of the topics
that we’re going to cover throughout the presentation. So, the effective classroom
observation – several different steps in that, and when we
think about observation, it’s not just the act of going
into a teacher’s classroom in and of itself and observing
the teacher’s practice. There are several key steps that
we have to make sure that we have in place in order for us
to really have that improved student learning. We’re seeing a
lot more feedback, collegial conversations
being held within schools, certainly between
administrators and the teachers, and then certainly among the
teachers themselves through the use of a lot of their PLCs
(professional learning communities) and certainly
common planning time, where they’re just really having
an opportunity to have the more quality conversations with one
another about student learning, and certainly from the
observer or the evaluator standpoint as well. And certainly, that feedback
is used to drive targeted professional learning for our
teachers and that is beneficial only when it’s
individualized to the teachers. So, when we go in and
we do these observations, it’s not about a mass or global
approach to the entire school, per se, of educators, but
certainly targeting the needs of each individual teacher so that
we can be sure that their areas of focus are addressed
for specifically what their needs are. And then, of course, by this
process all tied together, ensuring that we have
each of those steps, we’re more likely to get to
that goal of improving our student learning. So, you know, certainly the
goal of effective classroom observation is to increase
that student learning. And when we follow these steps
and procedures and processes of a solid observation
process or cycle, then we’re more apt to get to
that student learning that we’re seeking to have. There are numerous kinds
of observations, certainly, that can be used. And I know there were a lot of
questions about different types of observations that came in
prior to the webinar today. So it’s great to see some of the
questions that you had kind of circulating around the different
types of observations and certainly what some
of these purposes are. The biggest thing that we
have to keep in mind is that observation is not a “gotcha”
or “caught ya” process. Sometimes how we build our
school culture is going to determine the exact success
that we’re going to have in an observation process,
or in the cycle. The formal observation in and of
itself, as I shared a while ago, is certainly not the only way
that we’re going to collect the data that we need. Formal observations, most
often, as far as length goes, 30 to 60 minutes, or
certainly a class period. It’s an opportunity for an
observer to see the beginning, the middle, and the end of a
lesson so that you get that full picture of how the teacher
introduces new content, provides that series of learning
and instructional strategies for the students, and then how they
wrap that up at the end and assess what the
students have learned. It, most often, is accompanied
by the pre- and the post-conference, and we’ll talk
about that in a little bit more depth in just a moment. The informal observation and
the walk-throughs are certainly something we’re seeing more of
as a result of the MET study. We found, of course, that – or
it was discovered that – more frequent and shorter
observations had a tendency to provide a better picture of
what was actually happening in classrooms, rather than taking
that one opportunity for a formal observation. Because as most of you
know, on many occasions, when we go in for that
formal observation, it has been something that
the teacher has had quite an opportunity to
rehearse, perhaps, even in the
preplanning piece of that. They’ve had, you know, time at
length to certainly plan and develop that lesson, and good
planning we know is certainly key to effective teaching. However, the formal observation
in and of itself can certainly make you feel as if you’re
getting that dog and pony show that we so often refer to it as. But it’s really the informal
observations that are unannounced and those
walk-throughs where we often gather the most evidence
that gives us a truer picture of the teaching. So, I know many districts are
using informal observations to help provide that truer picture
– those unannounced visits when they can go in and gather
evidence to support the areas within the domains
that are being observed. And then the walk-throughs are
most often used, of course, to support the observation
findings previously or to add to, to get a better picture of
the summative evaluation that comes at the end of the year. Now, one thing that I’ve noticed
recently with the walk-throughs is that there’s certainly been
kind of an interpretation that leans more toward the
inappropriate side, just from the standpoint
that teachers, you know, will view that as an
inspection – “snoopervision,” as I’ve heard it
called by some educators. When the principals or
administrators/evaluators walk through for those impromptu
walk-through observations, they say, “It’s
snoopervision day,” if they know the
administrators are out and about. So, we have to work to eliminate
that feeling of “snoopervision” or that teachers
are being inspected. Checking on, making sure that
they’re doing what they’re supposed to be doing, but it’s
got to be more of a means of supporting and motivating our
teachers and encouraging them, because it’s all about
reflecting on their practices. So, rather than that inspection,
which will often stop teachers from wanting to have those
collegial conversations and seek to grow professionally, we have
to switch that culture [unclear] environment to understand
that this is an opportunity to motivate our group of teachers,
and to really support them, and to help them learn to
reflect more on their teaching practices as we’ve
never done before. And I was glad to see a while
ago that, in the survey, there was a certain percentage
– I believe we had about 38 percent say that there had
been more opportunities for conversations
around instruction. But that certainly is a
percentage that we would like to see increase, because that is
really one of the areas that helps to bring I think that
support and collegial atmosphere to the observation process
and the opportunity for professional growth. And we’ll certainly address
feedback a little bit more here shortly as well. So, thinking back to what I said
a moment ago about the formal observation process, this is the
one traditionally where we do have that opportunity to meet
with the teacher on the front end, and find a little bit out
about what exactly the lesson is going to entail and to which
part of their curriculum that lesson relates. We’re looking for those
connections with their standards, and how that
particular learning that we’re going to be hearing about on
that particular day fits into the sequence of
learning for that class. It’s an opportunity to learn
about the environment and the culture of the students that
[unclear] and what type of differentiated
instruction we can expect. So, that gives us the
information on the front end to do a better job of observing
in the classroom once we know pretty much how the teacher
is planning to handle certain groups of students. And, of course, the observation
in and of itself – we’re going in, we’re collecting
all of our information, and thinking about that
observation information in terms of a cause and
effect relationship. Because the teacher is
saying or doing one thing, the students are learning
something as a result of that. And that is truly what that
observation is about – what the students are learning. So, because of what
the teacher is doing, what kind of student
learning and to what degree are we seeing? And then, of course, the
post-observation conference we know is our opportunity to
have that reflective type conversation with the teacher
and talk about strengths and areas of improvement that
are going to be needed, and the next steps which are
crucial to providing a teacher. Because it’s not enough to
certainly go in and say, “This is what went well,”
or, “This is what we need to improve on.” But they need to know they’ve
got that support and opportunity from their administrator to help
them get to that improvement. So, when we think about the
target outcomes of effective classroom observation, the key
piece here is what I think has held us back, as administrators
are sharing with me throughout the U.S., is that there hasn’t
been that shared understanding of what good teaching
actually looks like. Teachers have not minded
so much being observed, given that they know what
they’re being observed on, and that they have that same
clear understanding of what good teaching and good instruction
actually looks like. So, that’s the piece that seems
to have been missing up to this point is just that common shared
understanding of what that effective instruction piece is. And certainly, of course, you
can continue to look through some of these that are here. The calibration piece – this has
been monumental in many of the districts that I’ve
been working with lately. Teachers – given the newness of
effective – the new effective evaluations or the processes
that states have been asked to develop – this probably has been
one of the areas that teachers have said have really helped to
give them that sense of trust from the people who
are observing them. They’re concerned very
much, or have been, about the training that
administrators or other evaluators have received that
would make them qualified for, say, if you will, to be the
“grader” of their instruction. So, the calibration piece that
many districts have opted to do where they have had training
with other administrators or evaluators on how to gauge
between one performance level and another, discerning whether
or not a teacher is needing improvement or
developing versus effective, or effective versus
highly effective. Having the chance to train,
really ascertain what each level looks like and sounds
like in a classroom, and then make those
ratings of a teacher. So, that formal calibration
process has brought a lot of trust to the process of
observation and then, of course, the assigning of those
summative evaluation ratings. So, many districts have been
saying this has been the best move that they have made,
and then – after the calibration process. And some districts are going
through recalibration year after year, have opted to do that to
keep that trust and regard for the process, especially with the
pay per performance issues that are so prevalent throughout
many districts and states now. So, they’re doing
the calibrations, there are
recalibrations [unclear], and then engaging in some deeper
trainings every two or three years in between that. The calibration process, when
you think about what quantifies, or qualifies, I guess, as a true
calibration process – and what’s the extent of training needed? We’re finding that usually,
at least a minimum of 10 observation practices is
a good start, for sure, for administrators who are
going to be evaluating, or evaluators who will be
observing teachers, you know, done through small
groups or individually, just really making sure that
there’s that emphasis on the quality of the evidence
that’s collected. Because a lot of times
we can be in classrooms, and I’m working with some
administrators now in one of the Florida counties who are having
a difficult time collecting the evidence of student learning
that actually connects to components and indicators within
their evaluation framework to accurately support the level
or the rating that’s given. So, I think that just making
sure that we have an adequate amount of practice observations
that these evaluators are taking place in, making sure that
there’s equality on the evidence collected, and not just enough
about – not just saying that we can make an accurate
rating for a teacher, but why and how are
we getting to that. Because that’s what teachers
want to most be assured of. If you’re telling them this –
they are needing improvement or developing, they want to know
the evidence that was collected. And it can’t be those opinions
about what we think or we feel or just kind of a bias, as well. So, we’ve got to keep that
connected to the evidence of student learning. Master coded videos are another
great way to increase the accuracy that we are most
often finding can certainly be inconsistent
throughout districts, and certainly throughout states. Master coded videos, though, are
not just videos that perhaps as a district, a group of
administrators or district staff members have gotten together
and have watched and rated on their own. When we talk about
master coded videos, we’re talking more about
videos, for example, from Charlotte Danielson through
her partnership with TeachScape that have been actually reviewed
and then master coded by her team of qualified researchers
who have certainly been trained and have worked to be able to
provide those ratings for the videos of lessons of teachers
teaching, as has Marzano. I know that Robert Marzano’s
team with LSI (Learning Sciences International) have certainly
[unclear] observation have worked to provide some of those
master coded videos as well. So, you have to make sure here
on this part that they truly are master coded in that they have
been done by someone of a higher authority than just a district
staff or group of administrators who have come to a consensus
perhaps about what a rating may be for a teacher. Rob, I believe this is the part
where you’re going to be picking up here and talking
about [unclear]. ROB RAMSDELL: Great,
thank you, Christy. And one of the resources that is
available in the middle of the screen, down at the bottom,
you should see a tab that says “Resources.” I know that at least one person
is having trouble downloading – has had some trouble
downloading the PowerPoint. I’m sure we can follow
up and get you that. But for those that are able
to access the resources, we’ve provided a self-assessment
tool that’s a resource that we’ve been developing over the
past several months based on input from a range of districts,
and we hope you’ll find thought provoking and useful. And the notion here is that
as we think about effective classroom observation and the
different activities that need to be put into place to ensure
that a school system has good capacity around effective
classroom observation, there are a number of indicators
that we can look at and reflect on as school systems. And so, we can – we’re going to
actually give you a quick poll and, again, for those
researchers that are on, I’d ask you again to try to
situate yourself in a district that you’re most familiar with. But an example of one
of the indicators on the self-assessment rubric is around
the issue that Christy was just sharing with us. And so the indicator is,
“Accreditation for observers is based on rigorous
competency requirements.” Now, we’ve only shown you level
2 and 4 in a 4-point rating that you might give each
of these indicators. So, just to say that we’re
going to ask in a moment to say whether you think your district
is at a 1, a 2, a 3, or a 4, but we’re only giving
you examples here of the 2 and 4 levels. So, if you think you’re
below 2, then you put a 1. If you think you’re
kind of between 2 and 4, of course you’d put a 3. But so, for a 2 level,
“Accreditation is based on attendance in training events
and does not require evidence of accuracy in rating
levels of performance.” And what we mean there
is accreditation of classroom observers. Whereas a level 4 is, “The
accreditation process consists of a range of assessments which
measure the trainee’s ability to implement all aspects of
the observation process. There is a strong emphasis on
accuracy in lesson observations, and there is a rigorous means of
assessing whether observers rate lessons accurately.” And the emphasis here on
accuracy is it’s not that we have a group of administrators
that are all agreeing with each other, because they may agree
with each other and all be wrong. Right? So, the notion of having some
anchor in a master coded video or with the support of experts
who have been accredited as master raters needs to be part
of the system if we’re committed to accuracy, not just
consistency of ratings. So, with that, we’re
going to pull up a poll, and if you can reflect on your
own experience in your district, or a district that
you’re familiar with, we’d be interested in
whether you view yourself as level 1, 2, 3, or 4. Great. I think we’ve kind of
settled in with a sense of the range of responses. We’ve almost – I guess around 74
percent of people feeling that they’re starting
at a level 1 or 2, with about a quarter of people
putting themselves at a 3 or 4. So, clearly that’s indicating
that there is work to be done around this issue of accrediting
observers and ensuring accuracy in the ratings of observers. Now, we’re going to go and
do a couple more of these. This is really useful to
have a sense of where the districts are. But I would also like you
to think of going through an exercise like this with a team
in your district as a way to get folks to reflect on the current
state of affairs and potential work together to develop a plan
for improving in these areas. So, the next indicator is, “The
training addresses giving and receiving feedback which leads
to more effective teaching.” So, “The training enables
observers” – this is at a level 2 – “to give feedback that is
mostly aligned to the rubric, but observers give teachers
few opportunities to respond or contribute to the discussion. While strengths and areas for
development are identified, there is no agreement on next
steps or further monitoring.” Where it’s fully developed, it’s
described that “The training enhances observers’
skills in giving feedback. The professional dialogue with
teachers is securely based on the components of the rubric. Where appropriate, the teacher
is invited to contribute to identify areas for
development and action steps, including further monitoring
to track progress.” So, again, we’re going to put up
a poll quickly and we ask you to indicate whether you feel
you’re starting at a 1, 2, 3, or 4 around the issue of
observers being able to provide good feedback. Great. I think we’re
kind of settling in. So, we have a little bit of a
different distribution here, with about 54
percent at a 1 or 2. And about 45
percent – of course, my math is not on particularly
strong display there, but we can all see the numbers
and have a sense of where people think they are now, and where
we may need to focus some additional energy. So, we’re going to look
at one more of these. This indicator
is, “Principal, assistant principals, and other observers have
opportunities for paired observations to maintain
inter-rater reliability.” And I’m just going to
let you read this one. So, I’ll give you a moment to
read the level 2 and level 4, and then we’ll do one last poll. Okay, I think we
can put the poll up. Okay, this appears to be the
area where folks on the call feel there’s the
most room for growth, with about 80 percent
scoring their district, or district they’re
familiar with, at a 1 or 2. So, this is really interesting
information and hopefully gives us all some food for thought as
we think about the work ahead in terms of improving
in these areas. We can go ahead
and hide that poll. In the resource that’s available
in the resources section, there’s a self-assessment tool. We have at the end
of it a scoring sheet. Again, so I’d like you to
imagine going through an exercise like we just went
through with the team in your district where you might compare
notes after asking individuals to rate the district and
identify common areas of strength and weakness, and where
there might be gaps in people’s perceptions about where the
district is strong or weak, with then a next step of moving
towards developing some form of action plan to improve in
these areas in a systematic and organized way. So, that resource is
available for you. Hopefully it’s helpful as a way
to kind of summarize a lot of what’s covered today
during the webinar, but also to maybe organize
some follow up conversations in your districts. I’m just going to touch
quickly on an issue related to reliability and some learnings
from the Gates Measures of Effective Teaching Project in
relation to observations and the combination of observations
needed in order to maximize the reliability of results, if the
results are being used as part of a summative
evaluation system. Now, this next slide is –
may feel a bit overwhelming. If you can just work with me
from left to right in a moment, rather than trying to
absorb it all simultaneously. The height of the bar and the
number in the bar represents levels of reliability based from
the Gates Measures of Effective Teaching Project research that
was conducted over the past three or four years. And we see that when a
single observer – so, starting from the left – when
a single observer observes one lesson, we have a
reliability level of 0.51. If we add a second observation,
but with the same person doing the observation, it does move up
in terms of level of reliability from 0.51 to 0.58. In the third bar, we still
have two observations, but now two different people
are conducting the observation, and we see a jump in the
reliability levels when it’s not only two observations but two
different people doing those observations. And then, as we
move to the right, we have a number of other
different configurations involving, for example,
in the fourth bar, a full observation by an
administrator and then three shorter observations
by pure observers. So, Christy was mentioning
what the Gates Foundation has introduced in terms of some
ideas about the impact on reliability of single
observations versus single longer observations versus
multiple observations but of a shorter length. And we see across here
that there’s a range of configurations that sort of
hover in a pretty similar territory, and as we think about
the implications of the number of observations required, if
we go all the way to the right, it may just not be practical. But this is a particular part of
the MET research that you might want to pursue as you think
about the kind of combination of observations and the different
configurations of people doing those observations
in your districts. So, mindful of time, I’m going
to skip over and get it back to Christy to dig in a bit on the
issue of feedback and improving the instructional conversations
and professional learning that’s connected to the
observation process. CHRISTY MCINNIS: Thank you, Rob. ELIZABETH GRENINGER:
Rob, this is Elizabeth. Christy, I’m sorry
– this is Elizabeth. I think there’s actually a few
questions about that slide. I think it might be helpful if
we address them before we move on, because I see some
questions coming in, if we can just – I’m going to
go back to that previous slide. Rob, I think people are asking
about – one in particular are the double observations of the
same lesson at the same time. ROB RAMSDELL: I believe that
those are two different lessons. ELIZABETH GRENINGER: Okay,
two different lessons by two different people. And the other question… ROB RAMSDELL: Right, so the
second and third bar – in one case, it’s two lessons and
the same person is doing it, and the third, it’s two lessons
and two different people are doing it. ELIZABETH GRENINGER: Okay. That makes sense. And then David is asking about
the five bars to the right. Are they
statistically different? ROB RAMSDELL: The five
bars to the right? ELIZABETH GRENINGER: If
you have an answer to that. ROB RAMSDELL: You mean the
range from 0.67 to 0.72? ELIZABETH GRENINGER: I believe
that’s what David’s asking. ROB RAMSDELL: Yeah. I mean, I think that most people
who have looked at this have looked at the bars
over to the right, and given the sort of impact
in terms of different levels of reliability and the realities
of what’s practical in terms of implementation, have sort of
concluded that that’s probably not necessary. Although, of course, anything
that can be done to increase reliability is important, right? But I think this kind of brings
us back to the whole issue of multiple measures. That is, we want for each of
these measures to get as high levels of reliability
as is possible, but within what is practical. And there’s a trade-off between
reliability and implementing something that’s feasible. So, most places that we’re
working with, I think, are seeing the multiple measures
as part of influencing decisions about how many
observations need to be done. ELIZABETH GRENINGER: Great. That’s really helpful and I
think you touched on a few other comments [unclear]. ROB RAMSDELL: Yes, Derek’s
question that just came up about the cost effectiveness of doing
this level of observation. It’s obviously something
that, at a policy level, needs to be understood as we
look at the impact of even just going from one observation
by one administrator to multiple observations. But Kelly has also brought up
concern around the lower levels of reliability and whether
you’re crossing an acceptable threshold given the high stakes
nature of this information or these scores if they’re being
used in an evaluative content with high stakes. So, these are, of course, the
issues that educators across the country are wrestling with,
and in the interest of time, I think we should probably move
on but maybe can come back to some additional questions on
this at the end if we have time. ELIZABETH GRENINGER: Great. Thanks, Rob. CHRISTY MCINNIS: Okay. Let me get us back to where
we were so we can go ahead and start talking about feedback. Again, as I mentioned earlier,
that observation in and of itself does not have a very
favorable long-term impact on teacher instruction. But it’s the other
components, as well, that add to the process and
certainly to the ability for teachers to make improvement
in their instruction. And certainly feedback – the
feedback is one of the key pieces that have the ability
to make that difference in teacher instruction. So, when we think about
conversations and professional learning, the quote here that
we’re seeing from the U.S. Department of Education, “To
have an effective teacher evaluation
system, you need good, trained evaluators and more time
from teachers and administrators to discuss performance and
improve teaching and learning.” And that really is the
crucial piece here. As you start thinking about
even what teachers do in their classroom and how feedback
impacts students’ performance, you can make those
same connections to the teacher learning. In classrooms, we expect
teachers to certainly do more than put a grade on
top of a student test. It’s that taking the
test, giving it back out, going over it with the students,
and talking about what they missed, what they did well, and
how that connects to the next phase of learning for them. So, the feedback piece
is crucial in learning. Likely, as well,
for the teachers. If we go into a classroom and
observe teachers and just come back out and say, “Great job,”
that non-specific feedback doesn’t really tell them what
was so good about what we’ve seen, and then certainly
where they need to improve. So, it’s just important that we
make sure we are fine tuning the feedback process. So, how are we taking that
information about what teachers are doing and using it as
a basis for improvement? There is great skill needed in
providing feedback or holding a feedback conference
with a teacher. The feedback in and of
itself should not be evaluative of a teacher. It’s more of a descriptive
conference and type of language that’s used to speak
with the teachers. Most often, I find in working
with administrators who are questioning about how to
start a feedback conference, I’ll share with them – have your
teacher, before they come in, think about things that
went well in their lesson, and things that
didn’t go so well. So that as the teacher comes
in, they have an opportunity to reflect and
respond to that first. The feedback is not just about
what we are sharing and giving to the teacher, but also about
their ability to reflect and certainly be able to participate
in that conversation. So, we’ve got a poll here, just
thinking about brainstorming about quality feedback. Quality feedback to
teachers has been limited. So, in a few words, I’d like to
ask you to share why you think that has been the case. And this is just a free type
– just share a few words, if you’d like, about why you
think that quality feedback to teachers has been so limited
throughout the years. Yeah, I’m seeing time
pop up quite a bit here. Yeah, time has certainly
been one of the big ones that many [unclear]. Lack of knowledge, I’m
seeing that as well. Lack of knowledge. You all are hitting on really
some of the top ones that I’m hearing from other districts
and other states for teachers. Sometimes there’s that
lack of instructional knowledge from the evaluator. Still, again, seeing
a lot of time issues, lack of collaboration, some
really great responses coming through of your observations
about that quality feedback having been limited. Okay. Yeah, and having hard
conversations – that’s certainly going to be another area that
we’re going to talk about. There is definitely a little bit
of that discomfort for some to provide that feedback to a
teacher, because most often, it does deal with having some
of the hard conversations, and oftentimes we find that
that’s just uncomfortable for administrators to do. As we rolled out the new teacher
evaluation processes throughout the U.S., you know, many
administrators stayed in their same schools. And I know many of the
evaluation systems that were previously used were a lot of
tally marks and checkboxes, and teachers predominantly
throughout the U.S. were rated satisfactory –
you either met expectations or you didn’t. So, now we’re coming back and
we’re having to switch the entire culture and the way that
this new evaluation is handled and the types of conversations
that it is requiring. So those hard conversations are
difficult to have because we’re not used to having had them. So, all right. Let’s go ahead and move on
and look at our next slide. So, why do we think that
quality of feedback is limited? And all of the things we’ve
listed here are many of the issues that you typed in just
a moment ago on that poll. What we’ve seen is that the
leadership has just been able to effectively deliver
that feedback and provide the support. And when we ask ourselves
why that is the case, and as many of you said, it’s
because of that limited training and knowledge about
instructional practices and the high quality teaching. So, in years past when we had
leaders who were more managers than they were
instructional leaders, we just didn’t have that
emphasis on the leadership as instructional
leadership as we do now. The accountability for student
learning just wasn’t at the level that it is now. Certainly, the competing demands
on time, all of you mentioned. And that’s where we have to
start looking at how we are using our time and managing
it and delegating it. And I know that even when
we have 48 hours in a day, we always need it seems like 72. Operational demands take
the front seat, and again, that’s just thinking of
administrators of the past and how it was more about being an
operational leader, a manager, rather than
looking at instruction. So, I like this quote from
Grant Wiggins because it is so applicable again,
as I shared earlier, to the teachers and the
feedback that they need. You know, when we think about
students in the classroom, it’s not just about the
advice we give them. It’s not just about
giving them grades. That isn’t what
constitutes the feedback. Nor does giving a teacher
just a rating or a form with a signature saying that
they’ve met expectations. We’re going to see the greatest
improvement in teachers based on the information shared during
that feedback conference. So, true feedback and how we
give it is what’s going to impact teacher
learning the most. And as someone shared a while
ago in their comments during that poll, it is
about the culture. How are we creating a
positive school culture, and are we from day one? And are we setting that
clear vision and mission, making sure that we’ve got
the clarity in our curriculum, and we’ve got high
quality assessment, good classroom
materials, technology, time for teachers to
engage in meaningful work, and smooth operations
throughout the school. But that support, and
that school culture, and letting teachers know that
administrators are right there with them in the trenches and
standing by them to support and help come up with ways to
improve their teaching is what is so crucial. Let’s look at this next poll… ROB RAMSDELL: Hey. This is Rob, and I’m
wondering if we can just pause here for a moment. There were a number of
comments and dialogue among the participants. And I think, particularly,
I wanted to acknowledge the comments people made about the
important benefits of paired observation and work in
teams around this process of observation and even the kind of
support that can follow up the observation that kind of goes
outside of the objective of trying to get as reliable
a rating as is possible. That is to say, what I was
interpreting people to share is that, of course, we want to
get reliable results for the summative measures, but there
also are important benefits in terms of leadership development
for the people that are doing the work of observations
together in the same classroom, in some instances, but also
involving even peer observers in the process together with
administrators and the benefits and building a culture of
trust and collaboration. I’m wondering if you can just
share some of your experience with that type of
work in the field, and if people have additional
comments about your own experiences, perhaps you can
share those in the content box to the left. CHRISTY MCINNIS:
Sure absolutely. You know, and that’s
a great point, Rob, a great conversation, because
now with many of the evaluation processes defined in districts
to include peer evaluators, some districts call
them peer mentors, I know there’s some difference –
there are some variances in how the terminology is used. But in many of the districts,
the same training that administrators, you know,
school principals and assistant principals are going through,
peer evaluators and mentors are going through the same training. And that is merely and most
importantly to ensure that we have a consistency in the
process and in the people who are providing ratings and
feedback to teachers to ensure that we’re all delivering the
same message and making sure that our ratings are aligning. And that’s really
important because, you know, where districts are investing
the money in peer evaluators, as we’ve learned from much of
the research that is ongoing right now, the multiple
observers bring great reliability to the process. So, having those peer evaluators
and mentors train with the school administrators and go
through the calibration process has really stepped up the
reliability in districts and the ability to provide
accurate ratings. So, that is something that
has been really increasing throughout many of the
states and the districts. And yes, it has been
a cost investment, but from what I’m learning
in working with many of these districts is that it is a
minimal cost compared to what could happen if we do not have
reliability and comparable information being
shared in a district, especially as we get closer to
that pay per performance piece. You know, the cost of that at
the end can be much greater than having the inaccuracy of
data within our districts. ROB RAMSDELL: I think another
comment that was shared earlier is the issue of needing to
provide training for teachers as well, so that they understand
what the effective practices are that they’re going to be
observed on through the observation process. And in the self-assessment
resource that we shared with you, that’s one of the
indicators in there that we didn’t cover during the polling. But again, Christy, I’m just
wondering from your experience if you could say a word or two
about the process that you’ve seen that works most effectively
in terms of engaging teachers in training activities that help
them understand the process, but through that training, also
provide better scaffolding for their own work to improve. CHRISTY MCINNIS: Yeah, and I
kind of – I got most of what you said I believe, Rob. It was a little
muffled on this end, but I think what I got out of
that was about the training for teachers as well. And yeah, it’s impossible to
train administrators without making sure that teachers
have received training that is certainly
equally as informative. Because they’re the ones who
feel like in this whole process that it’s being done to
them instead of with them. So, we have been working very
closely with many districts to also train their teachers. In training
administrators and evaluators, we’ve been asked to come back
and provide that same training and effective classroom
observation with many of the teachers as well. Because sometimes they certainly
want to hear it from someone else, other than
their own district. So, teachers have to have a
quality and level of training to really feel like they’re more
knowledgeable in the process and that they are ready, not
to compete, if you will, with the administrators
and evaluators, but that they can do what is
expected of them in terms of effective
instruction for students. So, yes, the training
for teachers is crucial. And that can be
done in various ways. Again, going back to – I know
Marzano and Danielson both have the rigorous training processes
built into their platform, and certainly we are doing a
lot of the face-to-face training with the teachers and
administrators as well. Does that kind of cover,
Rob, what you had asked me? ROB RAMSDELL: Yeah, you got it. So, thank you for that, and
maybe we can go to the poll. Now, this poll is a little
bit redundant maybe with the previous poll, given
particularly the focus that many of you had in your responses. But sort of moving back
to the issue of feedback, imagining that the observation
process is successful and we’ve built the capacity
around good observation, and they’re in that moment where
there’s a conversation occurring between observer and teacher. Oftentimes we see and have
experienced even ourselves as teachers that there’s a
hesitation to actually deliver feedback about areas
that need to be improved. And in the same way
as the previous poll, we’d like you to just share a
few words about what you think it is in that moment between
teacher and observer that gets in the way of delivering
honest feedback focused on areas of improvement. Excellent. CHRISTY MCINNIS: Yeah, these are
some really great comments that are coming in. And, yeah, and so
as you’re typing, I’m just going to make a
couple of comments just for the sake of time. The lack of
confidence I’m seeing, fear of hurting others’ feelings
– it’s most important in these feedback conferences that we are
making sure that the language we use is not personal language. If we keep our comments
tied to student learning, and practices that impact
student learning favorably or negatively, the conferences tend
to go better than some expect. Because most teachers,
at the end of the day, they want their students
to learn and do better. So, making sure that our
language is not used as a personal attack, if you will,
but more that it centers around student learning. Yeah, other comments coming
through – lack of pedagogically strong observers, a thought
that it’s tied to your job. And in reading many of these
again I just have to come back to, at the end of the day, as
administrators and evaluators, again we have to remember what
our number one responsibility is and it’s to the student learning
– the students in our schools. So, knowing that principals have
the greatest impact on teachers and their instruction, and that
teachers have the largest impact on student learning is just
certainly a direction and a point to remember. ROB RAMSDELL: Hey, Christy –
Derek has just asked a question. Your emphasis on the focus not
just being on teacher actions, but on the relationship between
teacher actions and student learning observed in the
classroom has sort of been a theme in your comments. And he’s asking if you can give
an example of feedback that is not about the teaching practice,
but about student learning. And, of course, I’ll just let
you speak to that and maybe clarify what you mean by making
sure that student learning is in the mix. CHRISTY MCINNIS:
Right, absolutely. And, you know, the
majority of, I believe, the states that are
listening in today, predominantly I believe are
using some version of Danielson – Charlotte
Danielson’s Framework. Certainly, there may be
some others out there. But, you know, when you are
thinking about feedback for student learning, I’m focusing
in a feedback conference around whether or not I gauged,
based on my evidence, there was student
learning that occurred. So, first of all, we’ve got to
have that ability to be able to recognize that before we
can even give the feedback. But I’m assessing –
looking at instruction, if I am speaking to a teacher
I might say, “I noticed after,” I’m just going to use this
as an example having been a previous English major, “I
noticed during teaching about verb conjugation that you really
didn’t stick with Jonathan as you were probing for his
understanding of how to conjugate the verb,
‘run.'” And I might say, “Can you share with me how you
think you could’ve handled that a little bit differently?” Or is something certainly
– well, most often, we’re not having these
conversations about what went really well in depth. We’re talking about that that’s
– more so we are just focused on where the weaknesses are. So, in assessing whether or
not the students learned, I’m going to come back
to how the teacher, and whether or not they were
flexible and responsive to whether or not
learning was taking place. And then, how they knew whether
or not the students were getting the information or certainly the
objective or central question of the day. So, if you come back to how the
teacher assessed the learning, that certainly is the feedback
about student learning. And Derek, I’m not sure if I’m
giving you the exact answer there you’re looking
for, but if not, please type back
in and let me know. But feedback about student
learning certainly has to center around how the teacher
assessed for that learning. Did they, or did they not? Okay, great, Derek. I’m glad we got that for you. Okay, so let’s move
on past this slide. Let’s see if we can. Professional learning. And again, this is the
end result of what happens with feedback. You know, if we are giving
feedback and we don’t connect it to next steps, then we’re
missing the boat of the purpose of feedback. So, as Charlotte Danielson
comments here about professional learning, saying that there’s
“a commitment to professional learning is important, not
because teaching is poor quality…but really because
it’s just so hard that we can always improve it.” And I make that point about
myself every day when I’m training and when I’m
working with administrators. You know, anyone in education
probably or should feel like we are lifelong learners. And certainly as educators and
expecting our students to be of such, we have to be able to know
that everything we do every day isn’t always perfect, and we can
always make improvements in what we do. And so, the best way for us as
administrators and evaluators to make that point is to live
it and breathe it every day. So, if we live it and breathe
it and we’re always in that learning mode and helping our
teachers understand that being in that learning mode and having
that constant drive to improve what we do, that’s what’s going
to make the difference at the end of the day in terms
of our instructions. So, as we link evaluation
and professional learning, and there certainly needs
to be a link because, again, the observation in and of
itself isn’t what’s going to be changing student learning. Making sure that we have those
standards for instruction, the high-quality standards,
whether it be Common Core, in Florida the
Sunshine State Standards, whatever may be being used
– making sure that they’re high-quality
standards for instruction. Using those multiple
standards-based measures of teacher effectiveness. We talked a lot today about the
teacher observation in and of itself as a means of
evaluating teachers. But we know, certainly, that
there are other things out there – making sure that student
perception surveys are used, multiple observers,
using value-added data, and certainly there are
many others there as well. High-quality training on the
standards, tools, and measures. Again, we have to put our
money where our mouth is. We can talk until the day’s end
about new teacher evaluation and the framework for using it, but
if we’re not training people in the tools to be the best
they can be in using them, we’re missing a part of this
process to prepare people to perform at the level expected. So, trained individuals to
interpret results and make professional
development recommendations. That’s what we have to [unclear]
as administrators and evaluators trained in the forums and
the processes we’re using, and making sure that the
professional development recommendations
that we’re making, based on the observations
we’ve made in a classroom, are things that will truly
impact teacher performance. Teachers need that opportunity
to grow professionally, and we can support
them, certainly, at the end of the day by
suggesting areas of improvement and making that happen for them. It’s not enough for us
to be able to say, “Okay, here’s where you’re weak –
struggling a little bit in the questioning area. Go see what you
can do about it.” We have to help provide
that in our district. How are they going
to get that training? And it certainly
has to be timely, as does that feedback
to a teacher. So, if we’re waiting two weeks
after an observation to feedback to a teacher, then
certainly we’re not giving that timely feedback. And what happens as a result
of that – the students are not getting the level of instruction
that they need because the teachers aren’t
making the improvement. So, this is kind of the synopsis
here of what we’re taking about, and linking the evaluation to
the professional learning must be individualized for teachers
because we know they all differ in their abilities in
delivering effective instruction in the classrooms. So, as we think about
bringing this here to an end, and I love this, “Where
observation – the rubber meets the road.” We have to step back
and take a hard look at instructional leadership. If we’re not about making sure
that we’re hiring leaders in our schools that are
instructional leaders, we need to step back and
look at our hiring processes. So in considering how we hire
our administrators and our school leaders, we need to be
thinking also about how the frameworks we’re using for
evaluation are used in that hiring process to ensure
that we’re getting the most qualified people. The feedback to teachers must
be as effective as are the expectations that we
expect teachers to have in their instruction. The effective feedback to
teachers is what’s going to help drive that ability and that
desire to want to improve. And then, certainly basis
for goal setting and professional learning. We’ve got to make sure that
we’re providing evidence of data collected and providing
goal-oriented comments back to the teacher, and not
just, “It wasn’t so great, here’s where you
did a great job,” but making sure that they
connect that information to effective instruction and to
the frameworks that we’re using. Just like telling a teacher –
and I’ll wrap it up with this – rather than saying
that, you know, “I like the way you greeted your
students as they came in the door,” it’s more about, “I
really like the way you greeted your students when they came
in the door because you made specific connections with
individual students and made them feel that you
really cared about them.” So, again, making sure specific
feedback that is tied to student learning, and that they know
that we’re right there in it with them, and we’re going to
support them to be the best that they can be. ELIZABETH GRENINGER:
Thanks so much, Christy. CHRISTY MCINNIS: Yes,
thank you, Elizabeth. Go ahead, I’m sorry. ELIZABETH GRENINGER: No problem. That was a lot of
information that we covered, and I know there have been
quite a lot of questions coming in and comments. So we’re going to reserve
some time for that. But before we jump into
the Q and A session, I wanted to take a moment to
find out from our audience, what action steps do you all
plan to take as a result of what you heard today? We saw throughout the webinar
some of the comments related to where you stand in your
districts and in your states currently with this
classroom observation piece. But now we’d like to know what
do you intend to do with the information you heard today? Looks like several folks are
planning to use this as part of their training and to
incorporate some of the ideas in training for observers. CHRISTY MCINNIS: I really like
– there are many comments coming through certainly about the
self-assessment tool, because, you know, we don’t know where
to go if we’re not assessing ourselves to see where we are. So, I think that’s
just a great start. ELIZABETH GRENINGER: I saw
several comments earlier, and again here, related to
teacher preparation and the way that higher ed can coordinate
and collaborate with our K-12 systems to ensure that we’re
all speaking the same language. And to understand what,
when teachers do get to the classroom, what the higher ed
folks can do to prepare them best for what they’re going to
find in terms of the evaluation and what they’re going
to be accountable for. CHRISTY MCINNIS: That’s
a great point, Elizabeth, and I would love to have a
moment just to chat about that, because actually that is one
of the trainings that I had certainly conducted for a lot
of teacher prep programs for various universities that were
feeding into one of the school districts, or the school
district that I had worked in. And because we were hiring so
many teachers who were coming from those teacher prep
programs in those schools, they came in already
having that knowledge. So, that is a great, great idea. I like having that brought up. That just puts them one
step ahead of the game, so that’s super. ELIZABETH GRENINGER:
That’s great, Christy. I think maybe that needs to be
happening in more places and it would help those novice teachers
be – to have that leg up when they enter the classroom. CHRISTY MCINNIS: Absolutely. I totally agree with that. Yes. I’d like to also – I appreciate
those of you who are making those connections with keeping
student learning as the central focus of observations and the
reason for doing what we do when it comes to observing
teachers and providing feedback. Keeping all of those
conversations and our purpose certainly centralizing
around student learning, because that is
what it’s all about. ELIZABETH GRENINGER: And Rob,
I see a few people commenting about the reliability aspect and
what you spoke about a little bit earlier. But do you have any concrete
suggestions for people to take away in reference to that, and
ensuring that they can go back and promote this
idea of inter-rater reliability among…? ROB RAMSDELL: Yeah, I was
thinking when – with some of the other questions that came up,
I think there must be a good resource that we could identify
that we might distribute out to the participants that would
provide a good summary overview. I don’t have something
immediately in mind, but it feels, based
on this theme, particularly in the comments
here and the final poll, that identifying a good resource
that might be useful as a reference point is something
that we should focus on, Elizabeth, as a follow-up. And then… ELIZABETH GRENINGER: Sure. I’ll just – go ahead. ROB RAMSDELL: I was just
going to say, again, if we’re incorrect about this
based on the bit of research we did about the states involved
here and many of your use of Danielson, certainly the
resources that Charlotte has been organizing with TeachScape
are something that should be looked at carefully to support
both the accuracy issue, but also to provide support
for teachers and to address the inter-rater reliability issue. So, hopefully people are
familiar with those resources that Charlotte, ETS (Educational
Testing Service) and TeachScape have been putting together
if you’re, in fact, largely
Danielson-focused districts. ELIZABETH GRENINGER:
That’s great, Rob. And what I was going to say is
that when you all – when we’re finished with the webinar
and you all go to the REL Mid-Atlantic website and go to
the page for this particular webinar, in about a week you’ll
see the resources associated with this webinar. So, you’ll see the
presentation from today, the self-assessment tool, but
we will also be compiling a frequently asked
questions document, and within that document there
will be a list of resources. So, we’ll be compiling any
resources that Rob and Christy have suggested today, as well
as some others that we’ll gather that will support some of the
questions that you’ve asked. So, be on the lookout for
that in the coming week. ROB RAMSDELL: It looks like John
has shared another one of the resources from the MET Project
that might be useful to share with other folks as well. Thanks, John. ELIZABETH GRENINGER: Great. So, it sounds like people have
a lot of work to do in this area and some really great and
concrete ideas for going back to your schools, going back to
your districts as researchers, going back to do the support
for schools and districts, and to really make this piece of
measuring teacher effectiveness one that is effective
and hones in on teachers, and then really gets at that
student learning that Christy has been very eagerly
promoting today. And at this point, we want to
take some time to answer some more of your questions. I know a lot of them came in
today through the webinar, and we got quite a few
in the registration prior to the webinar. So we want to definitely take
the opportunity to address some of those over the
next 15 minutes or so. If you found that the question
you asked on the left-hand side in the content box hasn’t
been addressed yet, please type it again into the
question and answer box that’s going to be coming
up on the screen, and we’ll also be pulling from
some of the pre-registration questions that came in. One question that I do know
came up, Rob and Christy, and you guys can decide
who wants to address it, but this idea that there may at
times be observers who are not familiar with the content area
in which they’re observing. What’s your perspective on
that, and how should those observations and feedback
discussions be structured so that both the observer
and the teacher can benefit from that experience? CHRISTY MCINNIS: I would love
to address that, Elizabeth. Thank you. You know, good instruction is
good instruction, first of all. And that’s something that we
do know and can recognize when we’re in a classroom. The issue, I think, that
some people will have is that sometimes, for example, you put
me in an AP calculus class and I’m probably going to be less
likely to know if a teacher makes a mistake
with the content. However, the pre-conference
really comes in handy in that respect, because I’m expecting
the teacher to share with me what they’re covering and how
it connects in the scheme of learning for those
particular students. I’ve often had conversations
around observations that have perhaps been at a content level
that was a little bit higher than perhaps what I studied
or what I was familiar with, just to sometimes clarify a
question about the content. But what I found is that
when it comes to instruction, good instructional practice and
strategy are the same across the board, and not – most often,
it does not come down to not knowing the content, as far as
impacting the ability to observe that particular lesson. ELIZABETH GRENINGER:
That’s great, Christy. That’s helpful. One of the other questions… ROB RAMSDELL: Most of the
observation rubrics, of course, that people are using
are not content specific. So, there are
examples, of course, of rubrics and observation
rubrics that are – have content – a content focus. But I think most of what people
are using are kind of agnostic on the specific content or
course that’s being delivered. I wanted to just touch on this
question that David has shared and I think what I’m
understanding in the question is this potential tension between
the reality of evaluation systems being implemented across
the country oftentimes with much higher stakes involved,
and at the same time, kind of the theme that
we’ve tried to emphasize, which is the point and outcome
of good evaluation systems should be professional
growth for educators. And Christy, I’m wondering if
you can speak in your experience about how you’ve seen districts
handling that tension and what may seem like opposing
objectives – one around holding somebody accountable and
increasing the stakes of accountability, while at the
same time trying to focus on growth and improvement? CHRISTY MCINNIS: Rob, were you
talking about that in relation to PD? I’m sorry, I actually clicked an
X and I believe I deleted that question on accident. So, can you… ROB RAMSDELL: Yes. Just the extent to which,
in your experience, this tension between the
emphasis on professional development versus the emphasis
on evaluation – how you – my initial slide with the seesaw
and trying to find that balance. Are those actually
at cross purposes? And how do we rationalize,
on the one hand, increasing accountability,
and on the other hand, trying to create a culture
that’s focused on growth and professional learning? CHRISTY MCINNIS: Well, I
think because many states and Departments of Education have
insisted that professional development programs be
increased and truly monitored and provide data that shows the
impact on student learning – I mean, we have to make sure
that there is accountability. And this is a question that
someone else was also raising as well about the
professional development, and how we assess that. There has to be
accountability, certainly, for the teacher performance in
the classroom because of student learning, but then we have
that responsibility as administrators, school leaders,
and certainly districts, to make sure that we’re
providing that research-based quality professional development
to help the teachers improve. So, it’s like giving them
one piece of the puzzle and expecting them to connect or
finish the puzzle without the other piece or pieces. So, we have an obligation to our
teachers to make sure that if we’re going to tell them
how they need to improve, or that they need to improve,
that we then have to be able to give them those strategies how
and then programs or processes to do that in manners that are
research based and proven to improve student learning. So, I certainly think we have
that responsibility to our teachers to provide that for
them if we’re going to expect the instruction to increase. ELIZABETH GRENINGER:
Christy and Rob, another question that I saw come
in was related to video – use of videos in observing. And what is your
experience with that, and have you seen it
being used effectively? ROB RAMSDELL: This is
an area where we have less direct experience. There are a number of
organizations that are doing a lot of work around the use of
video in the observation process in terms of actually, in the
capturing lessons through video instead of live observations. We’re probably not in the best
position to speak to that, but again, maybe you could
follow up with some resources and links to
organizations that are focused on that in particular. Of course, there is then the use
of video to support the training process which is – and to
support the accreditation process, for example,
through master coded videos. That’s an area where we
have more experience, but I think it was the former
that you might’ve been asking about, and I think we should
follow up with some examples of resources and organizations that
have focused in on the use of video rather than
live observations. ELIZABETH GRENINGER: Yeah,
great – that’s great, Rob. We’ll definitely
investigate that and include that in the FAQ document. CHRISTY MCINNIS: Elizabeth, can
I address a question that Derek has here about data that
we use to assess PD? Because I know so often that we
don’t make decisions about what we’ve done in schools and
whether or not it’s working until we get students’ test
scores at the end of the year, and sometimes those aren’t
coming in until the beginning of the next school year. So, this is where it is just
imperative to have continuous presence in classrooms to
make sure that we are seeing strategies being used and
implemented that we are working with our teachers on, whether it
be in school-wide professional development, whether it be
strategies they’re using and working on in their PLC. But that’s where our presence in
the classrooms have to be made more frequent, so that we can
monitor how those new strategies and practices are being used and
whether or not we’re seeing an impact on student learning. And I know with a lot of the
data chats that are taking place in more and more districts,
there seems to be more of a monitoring of that
data on a week-to-week, month-to-month basis, to see
what the impacts are and the effects on student learning,
based on the new strategies and PD that teachers are going
through and implementing. ROB RAMSDELL: So, Alica has
just shared some important information about an extension
to the MET Project where 15,000 lessons have been
videoed, it sounds like, to create a library of
practice, and will be available in early 2014. So, it sounds like that’s
certainly something we should stay tuned about. And I want to look at Evonne’s
question here where she asked, “Are there programs that include
self-evaluations that teachers could use? This could be less threatening.” And, Christy, I don’t know if
you can speak at all to how you’ve seen teachers using
rubrics like the Framework for Teaching on their own – not
necessarily just in response to the evaluation process. CHRISTY MCINNIS: I know
that there are, again, many districts who ask teachers
to self-assess after an observation, and I know that
those are used, certainly, in those feedback conferences
just to have talking points so the teachers can begin
to see where, you know, their interpretation of
what’s effective or what’s not effective. And then certainly the
administrator or peer evaluator who’s been calibrated where they
can see where those areas are. That, when that is
used in that process, that as well is not a
threatening process. Just, I think that conversation
and getting teachers to begin self-assessing, self-evaluating,
having a buddy teacher come in and watch their teaching and
provide feedback for them. And I know a while ago certainly
the video piece was brought up. But teachers – there are many
teachers who have started to video themselves and watch the
video back to have that – to self-assess their own teaching. It’s kind of that same premise. You know, you hear yourself
sometimes at the end of a video recorder or on a cassette tape
or on a voicemail and you think to yourself, “Is that
what I really sound like?” And it’s the same thing with
watching yourself teach. We don’t always recognize and
observe what we do when we’re in the moment doing it. But as far as a program in
and of itself that includes a self-evaluation, I can’t think
of one just right off the top of my head. ELIZABETH GRENINGER:
And think, Christy, the big picture with
that is even, you know, our best teachers who want
to do that and do their self-assessments and engage
in peer observations, the challenge again
is time – what everyone talked about earlier. And so, finding the time to
do that and making that the priority I think is where that
becomes challenging for folks. CHRISTY MCINNIS: Right. And I totally
understand that time piece. I know that with the use of all
kinds of technology these days and handheld devices, I
know it certainly, you know, there’s certainly more – there
are more opportunities that are readily available
to make that happen. But then I know you certainly
have the time to go back and watch something like that, or
the time that it does take to self-assess. But again, this is where we
start really focusing on that culture that we’re building
within our buildings to make sure that teachers see the value
in that professional growth, so some of that has to be
started at that level just in culture and
environment within a building. ROB RAMSDELL: I see Aaron’s
question about specific tips for private schools, and I think
much of – hopefully much of what we’ve talked about today will
be – is relevant and applicable. Of course, we oftentimes are
referring to district and state policy, which may not apply in
the same way to private schools, but the principles of improving
observation in order to inform and serve as a catalyst for
growth and improvement I think applies similarly in
whatever setting we happen to be teaching in. ELIZABETH GRENINGER:
Thanks, Rob. I just want to wrap up
with one last question, and this relates to
administrators and their evaluations, and thinking about
how effective administrators are at doing these observations – conducting
effective observations. How have you seen this play into
administrators’ evaluation for schools and districts, and
how they’re being rated? ROB RAMSDELL: Christy, do
you want to take that one? CHRISTY MCINNIS: I’m sorry,
Rob, did you want me to take it? ROB RAMSDELL: Yeah. CHRISTY MCINNIS: Okay, thanks. I’m sorry. Didn’t know if I was talking
over you, I apologize. It certainly is playing into the
new administrator evaluations because certainly part of what
administrators are expected to know and do is to be able to
observe effective instruction and to share effective
instructional practices and strategies with their teachers
to improve student learning. So, their skills and abilities
in doing all of these different pieces of the teacher
observation are part of what they’re being assessed by on
their administrator evaluations. So I know just recently we’ve
been doing a lot of work with districts on the principal
school leader evaluations and helping them be able to ensure
that they’re certainly getting the best out of their
evaluations that they can, because they are very much held
accountable for the learning in their building for students,
and certainly that level of teacher instruction. So, there’s a big
connection between the two, and as we are moving into
some states’ second, third, fourth year in the
teacher evaluation, they’re now really starting to
turn and take a look at that leadership assessment as well. ELIZABETH GRENINGER: Super. Thanks, Christy,
for touching on that. I’d like now to just give a
virtual round of applause to Rob and Christy for taking the time
out today and sharing their valuable expertise in
classroom observations. It’s clear that people are
interested in this issue and there’s a lot of work to do, so
we do hope that the information you’ve heard is useful, and that
the continued conversations at our forum and the resources
we’ll be able to provide to you will be useful in your practice. And you can see here on
the slide, right now, Cambridge Education’s website if
you want to find out more about the work they’re doing. And you’ll see our
personal contact information. If you do have any questions
about this webinar or any of our future webinars, please
do reach out to any of us. I’ll also point out that when
you do your feedback survey today, there will be an
opportunity to with your feedback in mind, so
please be honest and provide us with input. At this time, we’re
going to be wrapping up, and we wish you all
a wonderful evening.

About James Carlton

Read All Posts By James Carlton

Leave a Reply

Your email address will not be published. Required fields are marked *