![]()
How administrators can evaluate programs beyond the pitch deck.
Most administrators who have been through a secondary reading intervention adoption know the pattern. The vendor presents Lexile gains, testimonials, and a glossy results brief. The committee scores the rubric. The board approves the purchase. Twelve months later, the data does not match the pitch. This is not always vendor dishonesty. It is usually a buyer’s framework that is not sharp enough to separate programs that work for adolescents from programs that were originally built for elementary and bolted onto a secondary use case.
This post lays out a practical buyer’s framework administrators can use during the next adoption cycle. It is built on three layers: evidence quality, design fit for adolescents, and operational viability in your buildings.
Layer 1: Evidence quality, with ESSA as the floor
Under the Every Student Succeeds Act, evidence for educational programs is sorted into four tiers. Tier 1 is "strong" evidence, requiring at least one well-designed and well-implemented experimental study with a sample of at least 350 students across two or more districts and a statistically significant positive effect. Tier 2 is "moderate," based on quasi-experimental designs. Tier 3 is "promising," based on correlational studies with controls. Tier 4 is "demonstrates a rationale," meaning a logic model exists but the evidence base is preliminary.
The What Works Clearinghouse maintains a searchable database of programs reviewed against these criteria, and Evidence for ESSA, run out of Johns Hopkins, provides independent ratings. Both are free, public, and available before the vendor walks in the room.
The questions administrators should be asking at this layer:
- What is the program’s highest ESSA tier rating, and which study supports it?
- Was the study conducted at the grade band you are buying for? Elementary evidence does not transfer to secondary by assumption.
- Was the study conducted with the student population you serve? Effects for striving readers, multilingual learners, and special education students often differ.
- Is the rating from the publisher’s own research arm, or from an independent reviewer like the WWC or Evidence for ESSA?
The What Works Clearinghouse explains the practical purpose of these tiers plainly: they exist so that districts can determine which programs, practices, and interventions actually work, for which students, in which contexts. The tiers are not a stamp of approval. They are a transparency mechanism.
Layer 2: Design fit for adolescents
Evidence quality is necessary but not sufficient. A program can have a Tier 1 ESSA rating and still be a poor fit for a sixteen-year-old. The second layer of the framework asks whether the program was actually built for adolescents.
The most useful diagnostic questions:
- Are the texts in the program age-respectful? Read three random sample passages and ask whether a real high school student would be comfortable being seen reading them.
- Does the program distinguish between decoding gaps and comprehension gaps in its design, or does it assume all struggling students need the same intervention?
- Does the program include morphology, academic vocabulary, and fluency as core components, not just comprehension strategy instruction?
- Does the program provide enough connected text for students to actually read at volume, or is it dominated by isolated skill exercises?
- Does the program treat multilingual learners as a distinct population, or are EL supports added on as translation only?
This layer is where many "research-based" programs fall apart. A program may have been validated on a study population that looks nothing like your students, with texts that no fourteen-year-old would voluntarily read, and a design that assumes a student gap that does not match the gap in your screening data.
Layer 3: Operational viability
The third layer is the one most often skipped in adoption committees and most often blamed in implementation reviews. A program that is technically excellent but operationally infeasible in your buildings will not move data.
Key operational questions:
- How many minutes per day and days per week does the program require? Does your master schedule actually allow that?
- What is the staffing requirement? Does it assume a trained reading specialist, a content teacher, or a paraprofessional? Do you have that staff?
- What does the professional learning model look like? Is initial training enough, or does sustained coaching come with the adoption?
- What is the progress-monitoring system? Does it generate data you can actually act on, in cycles short enough to inform instruction?
- What is the per-pupil cost over a three-year window, including consumables, technology, and PD? Sticker price is not real cost.
Three red flags worth taking seriously
Beyond the framework, a few patterns reliably predict adoption regret.
First, an over-reliance on Lexile gains as the headline outcome. Lexile is a useful but limited measure, and it can be moved by passage selection and test-taking strategy without genuine reading growth. Ask for state assessment data, not just internal Lexile data.
Second, a program that does not provide enough text. Reading volume is one of the most consistent predictors of reading growth in the engagement research. A program built around two short passages per week will not produce volume, regardless of how rigorous the comprehension strategy work is.
Third, a program that does not adapt to a striving adolescent’s reading level without infantilizing the content. This is the practical reason hi-lo catalogs, including Saddleback’s, often end up alongside core intervention programs in strong secondary plans. Striving readers need access to text they can decode and content they want to read, and that combination is hard to find inside a single intervention program.
What administrators can do this year
If a major adoption is on the horizon, three moves will sharpen the process.
- Pull the ESSA evidence rating and WWC review for every program on the shortlist before the vendor presentations begin.
- Build a three-passage sample review into the rubric, scored by ELA teachers and ideally by a panel of students.
- Run an operational feasibility check against your actual master schedule and staffing model before signing.
The bottom line
Secondary reading intervention adoption deserves more than a rubric and a sales meeting. A serious buyer’s framework asks three questions in sequence: Is the evidence strong, independent, and matched to my students? Is the design built for adolescents, not bolted on from elementary? Can my buildings actually run it? Programs that fail any of the three will not move your data, regardless of how good the pitch deck looks.
Sources
- Institute of Education Sciences. ESSA Tiers of Evidence. https://ies.ed.gov/ncee/wwc/essa
- Institute of Education Sciences. What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/
- Evidence for ESSA, Johns Hopkins University. https://www.evidenceforessa.org/
- Institute of Education Sciences. Improving Adolescent Literacy: Effective Classroom and Intervention Practices. https://ies.ed.gov/ncee/wwc/practiceguide/8
- Saddleback Educational Publishing. Hi-Lo Solutions for Striving Readers. https://www.sdlback.com/