Are Your Assessments Valid for Your School Improvement Efforts?

As part of their district’s unified improvement plan, teachers are sometimes required to use data from assessments. Teachers gather periodically to review results of a recent assessment to discuss the results and what to do about them. Sometimes the conversation goes something like this.

“Wow, these kids continue to struggle.”
“Well, these assessments don’t align with what we are teaching anyway.”
“Why do they make us give these tests?”
“They help the kids prepare for the state test.”

The conversation is frustrating. While teachers are supposed to be using the assessment results to improve our instruction, what we really get is less instructional time, and precious little information to help them understand students and their response to instruction.

Are the assessments chosen for your school improvement plan valid for the ways you intend to use them?

Nearly every assessment provider will claim that their assessments are “valid and reliable.” But it is critical to consider whether your assessments are valid for your purposes. Are the assessments chosen for your school improvement plan valid for the ways you intend to use them?

What is Validity in Assessment?

Validity in assessment has two aspects, both are critical for an assessment to be valid.

• The assessment measures what it intends to measure.
• The results of the assessment are used for appropriate purposes.

A classic example to help people understand validity is assessments of foot size. Shoe size is a valid measure of foot size. Trying on shoes to see if they fit well, is a valid way to measure the size of a foot. An appropriate use of that assessment would be selecting which shoes to purchase. A valid assessment used for a valid purpose.

However, if you use that same foot size assessment to determine reading groups, or reading performance, it would not be valid. This is true in spite of the strong correlation that exists between foot size and reading ability, simply due to the fact that young children have smaller feet and usually do not read as well as older kids.

In order for an assessment to be valid an assessment must both measure what it intends to measure, and be used for appropriate purposes.

Valid Assessments for School Improvement

One of the most important purposes for educational assessment is to inform school improvement efforts. Teachers use assessments formatively, to guide planning, and inform instructional decision making. Schools use assessment results for their PLC (Professional Learning Communities) and DDI (Data Driven Instruction) work, to identify students who are struggling, and form groups for intervention. Leaders in education have long seen assessments as a critical component of school improvement, both for informing improvement efforts and for monitoring success.

What does it mean for an assessment to be valid for school improvement? Once again, it depends on how the assessment is being used and how you intend to improve schools.

Improved Instruction is Essential for School Improvement

An essential element of school improvement is improved instruction. If you want better learning outcomes, better instruction is a good way to get there. The PLC and DDI movements center assessment data in their processes. Curriculum writers and publishers seek to write better lessons and create materials for better instruction. Generally educators agree, the most effective teachers are those who teach better. If we want to improve schools, improving instruction is necessary.

How does instruction improve? Some suggest that excellent curricular programs are the solution. Yes, some materials are clearly better than others, but anyone who has worked in schools knows that those materials in the hands of highly competent educators will deliver results, and in the hands of a different teacher those materials will get very different results.

The real keys are in the head of the teacher themselves. In order for instruction to improve, teachers need to teach better, regardless of the curricular materials and tools they have been provided with. What types of assessments help teachers to reflect on, and improve instructional planning and delivery?

Three Aspects of Teacher Learning for the Improvement of Instruction

Students: Teachers need to understand their students. They need to understand what they know, and what they need to learn, but understanding students is more than that. When teachers know how students process and learn new content, they learn how to teach them. This includes understanding engagement, motivation, prior knowledge, skill levels, perspectives, and more. Humans are infinitely complex, and when we better understand our students, we are better able to meet their needs and take their learning to the next level.

Pedagogy: This is the technique of teaching children; the teachers’ craft. Pedagogy is lesson plans, instructional delivery methods, and all the intricacies of conducting highly effective learning opportunities day after day.

Content: This is the subject matter being assessed: mathematics, world languages, writing, phonics, etc. Teachers need deep understanding of subject matter in order to teach it well.

If the purpose of an assessment is to improve instruction, the key question is, how well does this assessment reveal each students’ current understanding of the material being taught?

Are You Really Assessing What They Tell You They are Assessing?

Before diving any deeper into this question of valid uses for assessment, we need to step back and ensure that we consider the basic tenet of valid assessments. Does the assessment assess what you are wanting to assess? Assessments should:

• Communicate meaningful information about what students know and understand relative to learning goals.
• Provide specific information about the skills to target for instruction.
• Provide evidence rich enough to help teachers identify areas of misunderstanding.
• Help teachers learn things about individual students and groups of students that will inform their lesson planning and instruction.

An Example of the Importance of Assessment Validity: Counting to 100

First grade students should learn a lot about numbers to 100. They should be fluent in their counting, by 1s and 10s, forward and backward. They should mentally be able to add ten to any number, and should be able to apply their understanding of numbers to 100 to add and subtract 2-digit numbers. The ability to count, and in particular knowing the verbal number word sequence, is critical.

Every state in this country lists in their standards and curriculum documents that students need to learn to count, and every instructional program on the market claims to help students learn to count. Additionally, those instructional programs come with assessments that have tasks designed to measure students’ counting ability. Counting is important. I would even go so far to suggest that if students cannot count, they cannot understand mathematics.

However, some schools and districts depend on computers to assess this critical skill. After students spend the allotted amount of time clicking through an interface a report will list whether a student is proficient or not on 1.NBT (as a domain which includes counting and a few other things) or perhaps even more specifically the standard 1.NBT.1 (replace with your state’s code) that directly calls for students to know the sequence.

Is there really a computer program that actually assesses a student’s ability to count? No. They use other activities, like sorting numbers, typing numbers, or selecting the next number as proxy measures. They assume that if a student can complete the missing number from a sequence like 5, 6, ___, 8 that they can count. (Notice the answer to this question is right on your keyboard.) They claim these as valid assessments of 1.NBT.

Some written tests attempt to measure counting by one in similar ways. Others resort to multiple choice, like in this example straight from an instructional program.

While the assessment guide might align this task with the standard 1.NBT.1 (Count to 120), does this task really assess the students’ ability to count? If a student answers this question correctly, we are not able to assume that the student to count to 120. And if the student gets this question incorrect, we cannot assume that they cannot count to 120.

The above question is not a valid measure of whether students have the ability to count. In fact, the only way to know if a student can count accurately is to listen to that student count. If the questions and the assessment methods are not a valid assessment of what they say they intend to assess, they are not useful, especially not for school improvement.

What Types of Assessment are Valid for School Improvement?

Improved instruction is essential for school improvement. When educators collaborate around assessment data, the assessment results need to serve the purpose of helping teachers to improve their planning and instruction.

There are 3 central characteristics of assessment that make them valid for supporting instructional improvement.

Aligned to Current Instruction – Giving a test on geometry while students have been studying place value is not helpful. Valid assessments must measure what they intend to measure, and if we want them to measure whether they are learning what is being taught, the questions need to mirror the content of current instruction. If teachers are going to learn what is working in their instruction, and what they need to improve, then measures of learning need to align directly with what they are teaching.

Detailed and Specific – Broad, high-level information about student growth and performance is sometimes helpful for systems leaders at the school, district, and state level. It is not helpful for teachers. To improve instruction, teachers need specificity. Let’s say a teacher is told that a student does not know all of the letters of the alphabet. This is not as helpful as knowing that the student knows 24 of the 26 letters. But, knowing that the student knows the names of all the letters but P and B is even more helpful. The more detailed the information teachers are able to get about their students, the more helpful it is for planning and instruction.

Transparent – When teachers are analyzing the assessment results they need to see the questions students are answering and the responses they gave. Many state and other online tests not only do not allow teachers to know which items were on the assessment. They also do not show how their students performed on the individual tasks. While assessments that are not transparent can serve important purposes in education, they are not valid for the improvement of instruction.

Are Your Assessments Valid for Your Efforts to Improve Instruction?

While schools everywhere want to capitalize on assessment data for their school improvement processes, not every assessment is valid for the uses they intend. When teachers gather to reflect on assessment results and to plan how they will respond, teachers need assessment results that are useful for that purpose. State assessment results, for example, are not valid for improving instruction. When leadership makes data discussions a priority, choosing assessments that will best support that work is critical.

Assessments that are highly valid and valuable for efforts to improve teaching and student learning outcomes are those that align with current instruction, provide rich, detailed information, and are fully transparent. These assessments are the most meaningful and helpful. Fortunately, these assessments are also often the least disruptive, and least expensive assessments teachers already use: classroom assessments.

About the Author:

David is an education insider with more than 25 years of experience in education as a classroom teacher, a district leader in math education, and a frequent presenter at conferences nationally. He founded Forefront Education to help educators better understand and share student learning information by visualizing meaningful assessment measures. David retired as elementary math specialist at Boulder Valley School District in Colorado in June 2020. Read on.