ÀÏ·ò×Ó´«Ã½

Marguerite Roza is the Director of the Edunomics Lab, which conducted the research, and a research professor at Georgetown University. JPR’s Jane Vaughan recently spoke with Roza about this research, what it means and how to increase student test scores.

Jane Vaughan: First off, tell me what is the Edunomics Lab?

Marguerite Roza: It's a research center at Georgetown. It's a made up word, Edunomics, a combination of education and economics. And we did that intentionally because our main focus is on all sort of the economic dimensions of education, how much is spent, where it goes, whether or not it is producing value for students, labor markets related to education and all of that.

JV: And then, how is that research used? What do you do once you've finished researching a certain topic?

MR: We do a lot of research. We try to make it very timely, very much in the moment, so that it can be useful for policy makers. We also use it in a lot of our trainings. We do a certificate in ed finance that's open to anyone. We have data displays on our website that states and districts can use in their budgeting and policy decisions. The idea is to put numbers out there so people can make decisions eyes wide open.

JV: You guys recently published some research into math and reading test scores across the country, looking at data from 2013 to 2024, and also looking at the amount of money spent in each state, public dollars that were spent on education. So first, I'm curious why you decided to research this topic in particular.

Marguerite Roza

MR: Edunomics Lab really has a focus that these public dollars that come into our system that we should leverage them to maximize value for students. Obviously, it's not sort of a one-to-one relationship. School districts do lots of things with the money, including transportation with kids and heat buildings and things like that. But we're always trying to look for ways to get more and more value out of each incremental dollar. And so we looked at these relationships across states to see which states had gotten more value, and some did. Places like Mississippi and Louisiana and Tennessee showed more improvement in their test scores over the same time period. Some states spent more money, some states spent less money, for various reasons. But our goal is to really learn from those that are successful and be able to use that information to propel all school systems forward.

JV: I want to talk about the results that you found in a second. But first, how did you conduct this research? Where are you getting your data, and how does that process work?

MR: As you know, every state has their own testing system. The problem when you just look at each state's individual, you can't walk it across states, and sometimes states change the test or the cutoffs from year to year. So those are useful on a very localized level, but every two years, we get assessment results from something called the NAEP, the , and it does a sample in each state. So it does not produce a score for each school or each district, but it does produce a score for each state, and it allows us to compare across states and over time, and we know that a lot of things happened in the last few years, so we wouldn't be expecting to see nothing but big growth between, say, 2019 and now. Obviously there was a pandemic in between. But this allows us to say, Where are we seeing more progress after the pandemic, where did we see more progress before the pandemic, and that's what we're trying to learn from in these comparisons. So those data came out two weeks ago. They were released publicly. We got them the same time everybody else did, and we lined them up with total spending per pupil over time in each state. So we were able to see what is the trend in the money, what's the trend in inflation, and then what are the trends in scores, and how do they compare across states?

JV: Okay, so you've got sort of a standardized test score that you can look across the country and compare more statewide, rather than, as you said, district specific.

MR: We're not able to call out which individual districts did better or worse from NAEP data alone.

JV: So broadly, looking across the country as a whole, what did you find in your research?

MR: Well, I mentioned already some states — Mississippi, Alabama, Tennessee — were some of the outliers on the stronger performance side. There are also ones that have been working really hard at it, and some states, including Oregon, have shown more decline over the same period, even as spending increased fairly rapidly over the same time period, more so than even in some other states. Oregon is not alone. There are the other states, like Washington state, for instance, that saw really big increase in spending alongside a decline in test scores, which is discouraging. One of the things we noticed in Oregon is that for one, the decline in test scores started before the pandemic. So even back starting in 2015, we started to see both math — and we look at math eighth grade scores and we look at reading fourth grade scores, and I can explain why in a minute — but we saw the decline starting before the pandemic. Obviously, there was a steep decline during the pandemic, but what is maybe most worrisome is that there is still continued decline when you look compared to spring 2022 to spring 2024, both in fourth grade reading and eighth grade math. So these scores are really kind of at a real low and continuing to fall, even as the funding is flowing in Oregon.

JV: And before we get specifically into Oregon and California, do you want to explain why you look at those specific scores?

MR: Sure, the fourth grade reading. It's an important score, because, as you know, most kids learn to read by fourth grade, and if they haven't figured out how to read, they have trouble being successful in the rest of school. And so that early reading is just really, really important, and we invest a lot of resources in that early reading and so what we're trying to do is capture is it working? There's been a lot of advances in how we do reading instruction in the last several years, and those are seen to be part of why some states are seeing a turnaround in those early reading scores. But that fourth grade reading is really important. Eighth grade math turns out to be a predictor of a lot of kids' opportunities in life. It's such a predictor of whether you graduate from high school, what opportunities you have after high school, whether it's in college, what kind of majors you choose, your lifetime earnings. Eighth grade math scores are a predictor of lifetime earnings, and they are a predictor of the state's future economy. These scores don't tell us everything, but they do tell us something, and they're an important bellwether that we need to be paying attention to because of their predictiveness for other kinds of things.

JV: Yeah, that's really interesting. So that data really has, potentially, a lot of impacts and a lot of predictors for the future, future economy, future outcomes for kids and for the state.

MR: As a parent, obviously math and reading scores aren't the only thing that matter to me. I want my kids to learn music and art and all those other kinds of things, but at the same time, when these scores are declining statewide, they are a concern.

JV: For sure. So let's dig into the specifics a little bit. You mentioned Oregon. Your research says that Oregon received about $1.7 billion in federal COVID money. But, as you're saying, as the funding has increased, it looks like test scores in Oregon have decreased as well over the last about 10 years.

MR: I'm not saying that that's so much like a causal effect, right? The money is increasing. The money did lots of things because remember when the money started really flowing, it was also when kids were coming back to school, and it was hard to hire teachers, and inflation was also increasing, which is a piece of this, it doesn't explain the full increase in spending. Teachers will widely say kids came back lacking some of the social skills that they had before, it was harder for them to be in class. These things are happening against the backdrop of social media and cell phones. There is a lot of coinciding things going on. So we wouldn't necessarily expect scores to have bounced back. We wouldn't see some sort of superhuman response in the money and scores. I think the concern really here is that the scores are still declining at a fairly rapid rate, and at a rate that's faster than what we're seeing in most other states. And that is the concern because the other states are experiencing those same contextual features that schools in Oregon are facing, too. And this is not a judgment of teachers, of parents, of individual schools or any of that, because there is a lot going on, but it is something that makes you pause and say, Are there ways that we could spend our money differently or that we could focus our attention differently to try to affect this pattern? Are we teaching using outdated strategies? Should we be updating our curriculum? Are we putting too much emphasis on some kind of programming at the expense of other kind of programming? This is just a point where we say, What can we learn from other states, and how can we try to change the trajectory in Oregon so that our schools can benefit and our students?

JV: And your data shows a similar trend in California as well. The spending has gone up over the last approximately 10 years, and the test scores have declined. And I just want to really emphasize this point that your data does go back to about 2013 so this is not necessarily just a COVID thing. I'm imagining people thinking, of course test scores are declining because of COVID, but your data is going farther back even than that.

MR: And I should point out that California didn't see quite the decade-long decline that Oregon did. So Oregon's decline over the decade is steeper than California's. Again, they're not apples to apples states either, but Oregon does stand out as steeper decline over the decade than many other states, including in California.

JV: And so do you have any answers? Does your research tell you anything in terms of why? Why is this funding not translating into higher student test scores?

MR: Well, what I'll first say is that I don't think most people were really surprised about which states had seen some of the greater progress. They are states that have focused first and foremost on academic recovery. That meant we focused on academic recovery more than all the other kinds of things we focus on. A lot of people will say, We really have to focus on social emotional health, or we really have to focus on culturally appropriate curriculum, or making sure that our kids are connected with extracurriculars. And it's also true that schools have been asked to focus on so many things that it is dizzying for the people who work in schools. And so I think in one sense, the states where we're seeing more progress have been unapologetic in their focus on the academic recovery first, not that they're neglecting everything else, but we really do social emotional health in service of teaching kids how to read and do math. So there was a very clear focus. The next thing I would say is they were unapologetic about the use of data. They don't make excuses for why scores have dropped. They don't say, Oh, kids had one bad day. They don't say things like, The test scores don't mean very much. They don't release their test scores on a Friday night of a three-day weekend and then hope nobody notices them. They really put them front and center and celebrated where they had successes and were really honest with parents and teachers and everybody where they didn't have successes. So I think that really dedicated look at the data was another piece. Another thing we saw was that some of the states that have seen more progress started some of these reading reforms earlier and took them very seriously. So, as many people know, the science behind reading instruction has really clarified that kids benefit from decoding skills, phonics-based reading, some of that early sound out your words kind of instruction, early on. So not come fourth grade, but really, like in kindergarten and first grade and second grade. And for any school systems that were teaching more the whole language model, which was like a guessing based approach to reading or three cueing or things like that, those were outdated models that were not working for many, many students. And so some school systems have said, Well, let's make sure that every single teacher is retrained early and put a state investment in that and done that early on and then chased down their data to say, We're not seeing progress in this classroom. Let's go make sure that we're not using outdated instruction in that classroom and done that. So really worked hard at revamping their core reading instruction programs from kindergarten through third grade. And that's paying off. I think that's one of those strategies paying off in some states where they really worked at that, not phased it in slowly over time. Said we're doing as much as we can today, and we're going to look at the data and figure out where it's not working and then go solve those problems. Tennessee is another state that has kind of made sure that any kid that was slipping behind got tutoring right away and made sure that school systems were offering tutoring and even summer programs into the summer and made that very clearly part of its formula. The money is for students who aren't successful in the school system. You gotta go chase them down and make sure that they get these services. Mississippi actually made a payment to every teacher that taught reading instruction to go get this what's called letters training on the side. So there are very concrete investments that were tied to things that were to deliver progress. And no looking away from the results. School systems very much feel like that's what they're focusing on first. There's no silver bullets here. The work is tough. There's a lot of work to do, and some of these states are just pushing harder on that focus.

JV: I'm curious about the sort of political undertones of what you're saying, that maybe states — Tennessee, Mississippi — tend to lean more red, more conservative, maybe were going back to school sooner since COVID than other blue states, like Oregon or California. Is that something that you see happening here?

MR: Well, I will say that political undertones don't explain everything. Florida is a state that also saw its scores go down. So if you were just trying to explain it all with red states/blue states, it doesn't explain at all. Then again, some of the loss in academic learning was tied to how long kids were out of school. And in some blue states, the kids were out of school longer. There's no way to go back and undo that now, but what we can do is try to fix what happened. The effects for kids will be permanent throughout their entire lives if we don't go really address that now. One thing that does seem to vary by state, and again, it's not perfectly a red state/blue state thing, is how much the state actually accepts that tests mean something. And you will hear some state leaders, I've not heard this in Oregon, but you will hear some state leaders say, Oh, the tests don't really matter. That's not what parents want. They don't care about whether their kid can solve an algebraic equation or anything like that. And I always think, Well, parents might care about it if we were honest with them about how predictive those test scores are for their kid's future or if we were honest with them how far their kids are behind right now. But years ago, the federal government was pushing on accountability, saying, We need a test for every school. We need a test for every district, and we will grade and judge every school and every district based on those tests, and there was a lot of pushback from that, often from teachers unions, but other groups as well, that it felt too punitive. And so the federal government backed off, and some states picked up that agenda of still trying to measure progress and use it to drive progress. And some states didn't. They just really walked away from caring about test scores. Generally, that tends to be more often in unionized states. But that's not the only variable there, so I don't think we can really attribute it fully to politics, but I do think there's an opportunity for state leaders who care about this stuff to come back with a lot of energy and try to solve the problem and really move the needle on their numbers.

JV: Talking about state leaders, what are you thinking in terms of potential solutions? What would help address this disparity that you're seeing in your research?

MR: In the end, there's no silver bullet. I don't think it's that easy. It's a slog. It takes a lot of work by a lot of people. We're going to try things. There's going to be things that don't work, and then we have to go back and fix what didn't work. You can put tutoring in place, and then find out that the kids aren't showing up for the tutoring, and then you have to solve that problem. You've got to solve each piece of the puzzle. The kids have to come to school. You have to figure out how to do that. And then when the kids are at school, we've got to maximize every hour and make sure that they're getting the value from these really strong reading programs. If your state doesn't have enough math teachers or good math teachers, and they're not delivering value, you may have to offer extra money to teach math in the state of Oregon or to teach math in a high poverty school in Oregon to really make sure that those investments and salaries are landing in those high value classrooms. And so you've got to figure out where there are the problems in the strategy, what isn't working, and then go in and solve those problems. So I think it's probably going to be a lot of things and a lot of data. It shouldn't be, Let's try something and sit back and wait two years until we get the next NAEP scores. It should be every quarter we're evaluating. Did the thing we tried work? Do we need to do a mid-course correction? And let's be honest with each other and talk to each other. Teachers will have a lot of answers for this. They'll know what stands in the way of their kids being successful. And we should listen to that system up and down the system to make sure we're fixing these problems.

JV: So if we have community members listening, parents, school board members ,who are concerned hearing about this, what would you recommend that they think about or focus on?

MR: What I think most people don't realize is school boards are the ones that own the fiduciary responsibility for how we spend the money in public education, and they might not feel like they do. The district prepares a budget and brings it to the board, but that is technically the spending moment. It's happening again this spring, and this is an opportunity for school boards to really say, How is our district doing? And they can look at their state test scores on that. Should we be changing the way we're spending money? What are we going to do in this next year's budget that's going to make sure that we see more success in math and reading, and have that conversation as part of those budget discussions. As for parents, the number one thing is, you want to know where your kids are academically on their state assessments. First of all, if you don't know, ask your teacher. How is my kid doing on the state assessments? Are we seeing progress with my student? What can I as a parent do to help? I know that the school will say really strong attendance is one of those things that can really help, making sure your kid's getting a good night's sleep, not staying up late on technology, making sure your kid is doing some reading, completing their homework, and are there any summer opportunities or tutoring opportunities that can help your kid get caught back up and on grade level, if they're not on grade level, because a student's future is very dependent on their kid's academic success, and you don't want to miss the next few years of chances to take advantage of all those opportunities if they exist.

JV: Yeah, that's really interesting. And so what about state lawmakers? They obviously have a lot of power here. What would you recommend that they be looking at?

MR: State lawmakers are the ones who are responsible for most of the money in public education. If they're going to spend more money, it's probably going to come from the state, for the most part, and I don't think they should give up on education. I think still investing more in public education makes sense. But I also think the urgency in the system around these results, around reading and math and making sure every kid is successful, that comes from the top. In Oregon, like other states, it would help if the lawmaker said, We're going to keep investing in education, and we really, really need to see progress on this. We're not going to make excuses for it. We want to see progress because we want to see Oregon kids succeed, and that conversation has to come from the top. And so kind of connecting those investments to those expectations for students — it's time.

JV: And you've talked about this a little bit, but just to put maybe too fine a point on it, what sort of effects does this have on the kids themselves, when there's all this money being spent and it looks like their test scores are not meeting the standards?

MR: There are real effects for kids. Individual students are going to be less successful the next year and the next year and the next year of school, if we don't get this turned around sooner rather than later. And schools are spending money on these recovery services, whether it's tutoring or summer programs or things like that. So you really want to make sure you're taking advantage of those. There might be things you can tap in locally in your neighborhood, whether there are tutoring programs or online systems and so on. But if not, we're closing kids out of opportunities to enter better paying jobs, to tap into opportunities to go to college or the military, be successful in future careers. It affects your lifetime earnings, even your life span. So I just don't want to underplay how important the academic side of schooling is. Even though we might feel more satisfied from going and watching a sports game or something else like that, we really got to attend to this now.

JV: I'm curious, from your perspective, if this research has sparked any additional questions for you. Do you have any potential future research that you want to do as a result of what you saw here?

MR: It sparked some reactions, I will say. And, yes, a lot of questions. I mean, we're really interested in what things people are trying work more than other things, and that's really hard to know. One of the things we know is that really strong, effective teachers, not surprisingly, are one of the most important things. When we got all these extra relief funds, some people got promoted out of the classroom to become reading coaches or interventionists or other kinds of specialists, and we went and hired new teachers, and in some cases, those new teachers, when the labor market was very thin, might not have been as strong as the teachers there before. We don't know. Should we be really luring these excellent teachers back to the classroom and trying to pay them more to keep them in the classroom so they're front facing with kids? That seems to be a likely scenario for this. I will say on the reaction side, I think some people thought we were judging them. That's not what we're trying to do. Some people thought we were saying, Huh, you don't need more money. That's not what we're saying at all. Or, You should return this money and not spend any more. That's not what we're saying at all. What we've been saying is make sure, as you get each incremental dollar, that you're finding ways to maximize the value for kids. And when you're not seeing it, go back and try again, right? Not give up. We're not making conclusions about the money. Some people have said, You should wait longer. Change takes time. Be patient. I don't accept that. I think this is a time for urgency, not patience. I just would want to emphasize that for us, this isn't a judgment of teachers, it's not a judgment of families, not even a judgment of schools or school systems. It's just really a call to action that let's figure this out for our kids and for this next generation.