March 14, 2025
For over 50 years, education policy has been guided by an insufficient understanding of economic disadvantage. One component involves poverty measures, as seen in the use of free and reduced-price meal data — National School Lunch Program eligibility (NSLP). These measures are used in the field of education to promote public policy and enable research and evaluation activities. For example, in Montana, Title 1 allocations are determined by eligibility data. District leaders track NSLP eligibility data in order to gauge how much Title 1 funding a district will receive. Since the incorporation of eligibility data into policy in the 1970’s, there have been questions as to which poverty measure to choose in what context. NSLP eligibility largely filled that gap. Nonetheless, NSLP eligibility data has many emerging insufficiencies, including over-identification of students, inaccurate income information, and inaccurate accounting of poor students in Community Eligible Provision districts (Geverdt & Nixon, 2018). During the pandemic, issues regarding eligibility data became more acute as the school meals program expanded and participation became decoupled from economic disadvantage. To face these challenges, alternative poverty measures have been proposed.
For this reason, we look to trends immediately before the pandemic (2019), as these most likely represent reliable historical trends with poverty measures and steps forward. There are criteria that the use of the NSLP data intends to fulfill (Geverdt & Nixon, 2018). Most districts and schools participate in the USDA program. However, the number of schools that do not participate in NSLP in Montana is significant (97 in March 2019). The NSLP program uses poverty data from the U.S. Department of Health and Human Services, which releases guidance in a frame that aligns with the school year.
This article focuses on seven alternative poverty measures. Alternative poverty measures include the Spatially Interpolated Demographic Estimates (SIDE) provided by the U.S. Department of Education and the Census Bureau. These estimates have as a target the school neighborhood. In this study, we use three SIDE measures that use American Community Survey data: the School Neighborhood Poverty Index; a measure created for this study based on school address; and a measure based on the geolocation of student addresses. A collection of income-to-poverty ratios for all identified students in a school could yield a granular school-level result based on the neighborhood in which students reside.
QUESTIONS TO CONSIDER
By comparing alternative poverty measures to the free and reduced meal data, we ask how correlated are the measures of school poverty to the NSLP measures for March 2019? Second, are the same schools classified similarly to the NSLP measure? Third, what is the impact of poverty measures on an analysis of student outcomes and institutional variables? The response highlights the impact of NSLP, allows an analysis of the relative strength of a poverty measure, and enables comparisons between measures. Fourth, the study looks at how variation in one outcome (satisfactory attendance) is explained by each student outcome or institutional variable when exchanging poverty measures as controls. Are there differences in the direction, sensitivity, and magnitude when using different estimates? Attendance is a common thread among schools, and satisfactory attendance is based on Every Student Succeeds Act requirements. By investigating the use case of Montana, we are able to see variation at a state, locale, and municipal level. This article focuses on state trends.
WHAT DO THE RESULTS SHOW
Overall, the most highly correlated poverty measures are NSLP participation and longevity. Participation is the count of those students actually enrolled in the school meals program, which is often different from the rate suggested by NSLP eligibility. Eligibility is based on the number of students eligible, a policy marker, rather than the actual enrollment in NSLP. The longevity measure is a construct of the number of years by grade level a student has participated in NSLP.
SIDE measures are highly correlated in a similar grouping. Of these, the SIDE estimates based on student address show the highest correlation. Small Area Income and Poverty Estimates (SAIPE) and direct certification data are moderately correlated. SAIPE is a census estimate of the count of students whose families live below the poverty line. Direct certification is the count of identified students in Community Eligibility Provision schools whose families receive federal benefits (SNAP and TANF).
To further measure the fidelity of each poverty measure with the NLSP data, we analyzed quartiles of the NSLP eligibility data compared to the quartiles of each poverty measure. This looks at whether schools with more students closest to the poverty level correspond with schools where students are mostly eligible for NSLP. Not surprisingly, the strongest matches were with direct certification and participation rates (Quartile 4). There is variation between quartiles for each poverty measure. This indicates that one measure may be more appropriate at some levels and not others. For example, SAIPE is more sensitive in quartiles 3 and 4, which represent higher counts of schools in the NSLP program.
When analyzing student outcome measures and institutional variables (attendance, graduation, dropout probability, dropout rate, achievement measures, elementary proficiency rates, ACT composite score, suspension data, teacher tenure, teacher salary, superintendent salary, and per pupil expenditure) by each poverty measure, we found that the NSLP eligibility data explained many student outcome and institutional variables to a greater degree than the alternative poverty measures. Direct certification and the SIDE estimate based on school address matched the magnitude of eligibility more reliably than participation and the other alternative poverty measures. SAIPE and longevity proved to explain little of the variation in student outcomes or institutional variables.
We then look at the sign, sensitivity, and magnitude of an analysis focused on satisfactory attendance. The magnitude of this was similar to the alternative poverty measures compared with the magnitude of the NSLP eligibility and the naïve condition. This confirms the finding of a RAND study, which found similar variation (Doan, S., Diliberti, M., Grant, D, 2022). There are important differences based on significance. For example, for the superintendent salary measure, the significance is stronger with the student SIDE measures than with either the eligibility condition or the naïve condition.
By noting differences in the same context, for example, by adding/removing an alternative poverty measure from the model, the study concludes that the use of a poverty measure is a choice dependent on policy factors. There are differences between how the measures correlate with NSLP eligibility, correspond to NSLP by quartile, explain variation in student outcome, and function in a model where all things are held equal except for the poverty measures. Nonetheless, no single alternative poverty measures have consistent findings that meet or exceed the magnitude of the NSLP measure. This impacts the ability of the measure to offer policy continuity.
This is seen with direct certification and other poverty measures. In fact, NSLP consistently explains more of the variation in the student outcome variables. The lack of consistency of the alternative poverty measures to meet or exceed NSLP eligibility values leads to the conclusion that decisions about using alternative poverty measures depend on the various constructs, policy or otherwise, of the poverty measures. An example of a construct is the value added when analyzing student neighborhoods by geolocating school or student addresses to derive an income estimate. By taking a granular approach, we can more readily identify differences and account for insufficiencies present in the NSLP data.
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R372A200011 to the Montana Office of Public Instruction. The opinions expressed are those of the author and do not represent views of the Institute, the U.S. Department of Education, or the Montana Office of Public Instruction.
Robin Clausen, Ph.D. (robin.clausen@mt.gov) is the Statewide Longitudinal Data System Program Manager with the Montana Office of Public Information.
References
Doan, S., Diliberti, M., Grant, D. (2022). Measuring School Poverty Matters, but How Should We Measure It: Comparing Results of Survey Analyses Conducted Using Various Measures of School Poverty. Rand Corporation: Santa Monica, CA. https://www.rand.org/pubs/working_papers/WRA168-1.html.
Geverdt. D., & Nixon, L. (2018). Sidestepping the Box: Designing a Supplemental Poverty Indicator for School Neighborhoods (NCES 2017-039). U.S. Department of Education. Washington, DC: National Center for Education Statistics. https://nces.ed.gov/programs/edge/docs/2017039.pdf