USNS Validity and Reliability

Forefront Education commissioned an external review of the Universal Screeners for Number Sense (USNS) by a team of researchers at Bowling Green State University.

Led by Jonathan Bostic, PhD., a professor and expert in areas of assessment and evaluation within mathematics education contexts and modeling with mathematics and model-eliciting activities; and Timothy Folger, MEd., a doctoral student in Leadership Studies in the College of Education and Human Development, the research team investigated the validity and reliability of the USNS assessments. Specifically, they examined whether the assessments reliably measure grade-level students’ number sense, whether items function effectively as a collective assessment tool, and whether the tools are effective for teachers to gather number sense performance information and monitor growth.

A consistent theme of both quantitative and qualitative results shows that the USNS assessments provide useful information about K-5 students’ number sense. The review revealed that USNS data can be used in ways to support instruction throughout an academic year and that USNS can be used to foster students’ learning and make instructional shifts to benefit learners. We are delighted to share the results of this review below.

Claim 1: Assessing Number Sense as Described by CCSS

The first claim was that, “The USNS assess K-5 students’ number sense in ways that are aligned to mathematics standards as described in the Common Core State Standards for Mathematics.” A content expert panel strongly agreed that the USNS reflects number sense topics described in the K-5 content standards. One panel member wrote:

“It [USNS] confirms students’ understanding of foundational concepts with place value, whole number operations and fractions that will allow the student to progress on to new grade level content in those domains.”

Variance was low, further indicating substantial agreement across the panel members.

Claim 2: The USNS Assessments Reliably Measure Number Sense

The second claim is the following, “The USNS assessments demonstrate effective construct development to reliably measure grade-level students’ number sense.”

The research team used a measure of consistency, Rasch reliability, which was excellent for grades 3-5 and good for grades K-2. Another reliability measure, called the Cronbach alpha reliability, was good to excellent across all USNS screeners. Access the full summary for data tables for these reliability measures. 

Claim 3: The USNS Assessment Items Function Effectively as a Collective Measure

The third claim was that, “The USNS assessment items function effectively in collectively measuring a single construct, students’ number sense.” The research team investigated whether the individual assessment items worked effectively together to measure number sense. Taken collectively, they validated this claim that the assessments effectively measure the single construct of number sense.

Individually, less than 5% of all grade-level assessment items were identified as potentially problematic. These results show that 9 assessment items were unproductive but not degrading to the collective measurement of number sense.

Claim 4: USNS are Useful to Gather Number Sense Information and Monitor Growth

The fourth and final claim that was examined was, “The USNS screeners are useful assessments for gathering information about K-5 students’ number sense. USNS screeners produce data that can be used to interpret growth between time periods (e.g., beginning of year to mid-year to end-of-year).”

A consistent theme resulting from qualitative analysis illuminated a finding that the USNS screeners provided useful information about K-5 students’ number sense. USNS data can be used in ways to support instruction throughout an academic year because they provide a measure of growth during the academic year. There exists strong alignment between intended and actual use, which indicates that USNS can be used effectively and safely to foster students’ learning and make instructional shifts to benefit learners.

If you wish to access the complete results of this study, please reach out to screeners@forefront.education.