If we can put a man on the moon….
Over Easter weekend Crystal and I took our kids to Washington D.C. The highlight of the trip may have been the planetarium at the National Air and Space Museum. We watched a film about planets in far-away solar systems similar to ours.
Here’s what’s fascinating about those planets: it’s impossible to see them, no matter how powerful the telescope. The film said trying to observe a planet orbiting a distant sun is like trying to see a firefly hovering near the lights of a football stadium from a thousand miles away.
But scientists know these planets exist - and can even estimate their size, temperature, mass, and orbital paths - by drawing inferences from the refraction of starlight passing around these invisible planets. It’s really an amazing testament to human ingenuity.
And all I could think about watching that film was: then why can’t we figure out what works in education?
If your response to that question is, “We can,” then you haven’t sat through a hearing of the Public Education Committee. On nearly every major question in education policy, policymakers are presented with conflicting opinions from experts and practitioners who have far greater expertise than we have, and we are forced to pick a point of view based on which experts we trust.
While the data that swirls around education policy can be inscrutable, it is imperative that we understand it better to inform our policy choices. I’ve spent a lot of time the past couple of weeks writing what has turned out to be a very lengthy series of posts for this blog about what I think we should do to improve public education in Texas. I’ll be posting them over the next several days - if I posted them all at once, no one would read it all (not that I’m sure anyone will read them one at a time either).
But before diving in, I think it’s important to review some data that I think are critical to understanding the policy choices we must make.
No, not that Simpson. Simpson’s paradox is something you have to understand to understand public education in Texas. Here’s why:
About a year ago I received a press release from the Texas Education Agency touting the outstanding performance of Texas students on the 4th grade science NAEP. The NAEP is a nationally-normed test taken by a statistical sample of students in all states, and is used to compare student performance from state to state.
On the fourth grade science NAEP, African American students in Texas scored 4th highest in the country among all African American students. Hispanic students in Texas scored 6th highest in the country among all Hispanic students. And Anglo students in Texas scored 8th highest in the country among all Anglo students.
And I thought, wow, thatis really good. Better than I expected. But then I saw another data point that didn’t seem to make sense. You know where we ranked as a state on the fourth grade science NAEP?
Now, how is it possible that the three student sub-populations comprising well over 90% of the students in Texas could each score in the top 10, but when you combine all of them, they’re 29th? I thought it was impossible, until Uri Treistman – a Ph.D. mathematician who happens to also be a global education policy expert – explained it to me.
Here’s how it works: nationally, and in Texas, African-American and Hispanic students underperform Anglo students on the NAEP. In Texas, however, African-American and Hispanic students comprise a much larger percentage of the total student population than in other states. Thus, even though each of our three major sub-groups ranks in the top 10within its sub-group, when you aggregate them, the fact that our lower-performing sub-groups constitute a larger percentage of the whole brings our overall average down to 29th. And this isn’t just true for 4th grade science NAEP scores - it’s generally the case that Texas minority and low-income students outperform their peer groups in other states, even as the state’s overall academic rankings range from mediocre to poor.
That’s Simpson’s Paradox. And it explains a lot about the Texas education system. If you want to evaluate the job that our schools and our teachers are doing, then judging by the disaggregated data we’re actually knocking the skin off the ball - and we’re doing it with significantly less per-pupil funding than most states. But if you want to know whether the public education system is fulfilling its mission to produce a workforce competent to the demands of a global economy and a citizenry competent to the complexities of a 21st century democracy - without regard to the color of their skin or the wealth of their families - by that measure, we are at best mediocre.
And this paradox - of a high-performing education system producing mediocre outcomes - will only grow worse with time if we don’t do something substantial to change course. The fastest growing segment of our student population is the segment that presents the greatest educational challenges. What that means is that even as our system operates more efficiently and effectively than nearly any other state, the educational hill our students must climb is steeper than in any other state, and will require even greater effort and investment.
I’ve been working on a lengthy post describing the education policies I believe it will take for us to achieve a system that truly lifts all students to reach their full potential. But first I want to post a bit more data that further highlights the issues presented by Simpson’s Paradox, and demonstrates the high performance of Texas schoolchildren on nationally normed tests.
Perhaps it should not be surprising that the source of this analysis is actually a physicist, given my analogy of the inscrutability of education data to the way astrophysicists infer the existence of planets orbiting distant suns. Michael Marder is a Physics professor at the University of Texas who co-founded the UTeach program and spends his spare time analyzing education data the way physicists analyze the behavior of particles. His analysis is fascinating, and I encourage everyone to review his detailed findings here. Below this post are some of the most interesting of his specific analyses.