Over Easter weekend Crystal and I took our kids to Washington D.C. The highlight of the trip may have been the planetarium at the National Air and Space Museum. We watched a film about planets in far-away solar systems similar to ours.
Here’s what’s fascinating about those planets: it’s impossible to see them, no matter how powerful the telescope. The film said trying to observe a planet orbiting a distant sun is like trying to see a firefly hovering near the lights of a football stadium from a thousand miles away.
But scientists know these planets exist - and can even estimate their size, temperature, mass, and orbital paths - by drawing inferences from the refraction of starlight passing around these invisible planets. It’s really an amazing testament to human ingenuity.
And all I could think about watching that film was: then why can’t we figure out what works in education?
If your response to that question is, “We can,” then you haven’t sat through a hearing of the Public Education Committee. On nearly every major question in education policy, policymakers are presented with conflicting opinions from experts and practitioners who have far greater expertise than we have, and we are forced to pick a point of view based on which experts we trust.
While the data that swirls around education policy can be inscrutable, it is imperative that we understand it better to inform our policy choices. I’ve spent a lot of time the past couple of weeks writing what has turned out to be a very lengthy series of posts for this blog about what I think we should do to improve public education in Texas. I’ll be posting them over the next several days - if I posted them all at once, no one would read it all (not that I’m sure anyone will read them one at a time either).
But before diving in, I think it’s important to review some data that I think are critical to understanding the policy choices we must make.
Simpson’s Paradox
No, not that Simpson. Simpson’s paradox is something you have to understand to understand public education in Texas. Here’s why:
About a year ago I received a press release from the Texas Education Agency touting the outstanding performance of Texas students on the 4th grade science NAEP. The NAEP is a nationally-normed test taken by a statistical sample of students in all states, and is used to compare student performance from state to state.
On the fourth grade science NAEP, African American students in Texas scored 4th highest in the country among all African American students. Hispanic students in Texas scored 6th highest in the country among all Hispanic students. And Anglo students in Texas scored 8th highest in the country among all Anglo students.
And I thought, wow, thatis really good. Better than I expected. But then I saw another data point that didn’t seem to make sense. You know where we ranked as a state on the fourth grade science NAEP?
29th.
Now, how is it possible that the three student sub-populations comprising well over 90% of the students in Texas could each score in the top 10, but when you combine all of them, they’re 29th? I thought it was impossible, until Uri Treistman – a Ph.D. mathematician who happens to also be a global education policy expert – explained it to me.
Here’s how it works: nationally, and in Texas, African-American and Hispanic students underperform Anglo students on the NAEP. In Texas, however, African-American and Hispanic students comprise a much larger percentage of the total student population than in other states. Thus, even though each of our three major sub-groups ranks in the top 10within its sub-group, when you aggregate them, the fact that our lower-performing sub-groups constitute a larger percentage of the whole brings our overall average down to 29th. And this isn’t just true for 4th grade science NAEP scores - it’s generally the case that Texas minority and low-income students outperform their peer groups in other states, even as the state’s overall academic rankings range from mediocre to poor.
That’s Simpson’s Paradox. And it explains a lot about the Texas education system. If you want to evaluate the job that our schools and our teachers are doing, then judging by the disaggregated data we’re actually knocking the skin off the ball - and we’re doing it with significantly less per-pupil funding than most states. But if you want to know whether the public education system is fulfilling its mission to produce a workforce competent to the demands of a global economy and a citizenry competent to the complexities of a 21st century democracy - without regard to the color of their skin or the wealth of their families - by that measure, we are at best mediocre.
And this paradox - of a high-performing education system producing mediocre outcomes - will only grow worse with time if we don’t do something substantial to change course. The fastest growing segment of our student population is the segment that presents the greatest educational challenges. What that means is that even as our system operates more efficiently and effectively than nearly any other state, the educational hill our students must climb is steeper than in any other state, and will require even greater effort and investment.
I’ve been working on a lengthy post describing the education policies I believe it will take for us to achieve a system that truly lifts all students to reach their full potential. But first I want to post a bit more data that further highlights the issues presented by Simpson’s Paradox, and demonstrates the high performance of Texas schoolchildren on nationally normed tests.
Perhaps it should not be surprising that the source of this analysis is actually a physicist, given my analogy of the inscrutability of education data to the way astrophysicists infer the existence of planets orbiting distant suns. Michael Marder is a Physics professor at the University of Texas who co-founded the UTeach program and spends his spare time analyzing education data the way physicists analyze the behavior of particles. His analysis is fascinating, and I encourage everyone to review his detailed findings here. Below this post are some of the most interesting of his specific analyses.
[video]
[video]
[video]
[video]
In yesterday’s blog post I promised that today I would post something much more important than why I’m against HB 5 - I promised to post what I am FOR. Unfortunately, I got sidetracked from writing that post because I’m trying to finish an op-ed solicited by the Dallas Morning News on HB 5, and it’s 12 minutes to midnight. Since family time for Easter starts tomorrow, the blog post will have to wait til next week. For those who can’t wait til then, there’s a sneak preview of my agenda for public education in the LBJ Future Forum video below. Happy Easter everyone!
[video]
I felt strongly enough about the shortcomings in HB 5 that I was one of only two members of the House voting against it yesterday. I felt strongly enough that normally I would have delivered a speech in opposition to the bill at the conclusion of the debate, but after 9 hours of debate, I don’t think anyone on the floor was interested in continuing the discussion, and it was clear it was not going to impact the outcome.
Still, I found that a lot of the floor discussion - and a lot of what I read online about testing - reflects a critical misunderstanding of testing’s track record in Texas. So I’ve decided to post here, for those who are interested, what I would have said if I’d given a floor speech against HB 5.
Illuminating the shadows
Six years ago, I had a discussion with a superintendent from an exurban school district that was rapidly growing and rapidly diversifying. This superintendent wanted me to know why school accountability - the system of evaluating student performance on standardized assessments and rating schools based on that performance - was such bad policy.
He told me how population growth in his community had transformed his previously rural, mostly middle-class, nearly all-white community so that now, on one campus, he had over 30 African American kids at one elementary school. The number 30 was significant because above 30 the accountability system required that the campus not only be evaluated on the achievement of all its students, but also specifically on the performance of the African American students on a disaggregated basis. And he said that the low test scores of these students put that campus, which had previously received very high ratings from the state, at risk of being labeled Academically Unacceptable. He told me that even though the rest of the students on the campus continued to do well, he was forced to marshal enormous resources to lift the performance of those African American students, to avoid a label that would jeopardize local property values, anger the local chamber of commerce, and provoke a major backlash from parents.
So I asked him, then what happened? Well, through enormous effort they had managed to lift the African American students’ scores up to avoid the label, he told me.
That’s supposed to be an argument against accountability? That seems like exactly the argument in favor of it. It’s not that this superintendent wouldn’t have wanted those kids to do well in the absence of accountability - it’s that he wouldn’t have had the same incentive to focus as much attention or resources on them without accountability.
And this is not an exceptional story; it’s the norm. I asked one of my colleagues on the floor yesterday, who I knew had a close relationship with a mutual friend of ours who works in special education, if he had discussed with our friend the possibility of weakening school accountability for the performance of special ed students on standardized tests. And he told me our friend had said “Don’t do that - it’s how I get the administration to support our department!”
Think of this phenomenon operating at scale. Now consider this: in Texas, since the implementation of accountability measured through disaggregated student data beginning in the early 1990’s, Texas 8th grade Hispanic students have gained 38 points on the NAEP math test - 14 points more than Hispanic students in the rest of the country - reducing the achievement gap by 22% relative to Anglo students in Texas. African American 8th graders have gained 43 points on the NAEP math test - 17 points more than African American students in the rest of the country - reducing the achievement gap by 29% relative to Anglo students in Texas.
And no, the correlation between those academic gains and the implementation of standards-based accountability does not prove a causal effect. But if you look at the timing of those gains in Texas, and then look at similar gains achieved by minority students in the rest of the country the following decade after NCLB introduced accountability in other states, it stretches the imagination not to see a strong connection. Look at these charts and note the introduction of our accountability system in the 90’s, and NCLB in 2001:
Not to say that all is well with testing in Texas
I have said for some time that we have lots of problems with testing in Texas.
None of these problems are endemic to standards-based accountability; all could be solved through common-sense improvements in the way we administer tests - which I have proposed.
My specific concerns about HB 5
There are some good things in HB 5. It gets rid of this terrible designation - the “Minimum High School Diploma” - in current law for students who opt out of the recommended, college-prep high school curriculum, and it bolsters the offerings under the new “Foundation” plan with better career preparatory offerings. But it does two things that I believe are going to have very negative consequences for the state:
1. It eliminates the current law that presumes that all students entering high school are capable of and want to pursue the most rigorous curriculum path, the path that affords them the widest range of opportunities in the future. On a couple of occasions Chairman Aycock argued that this “Distinguished” diploma path (as newly renamed by HB 5) was not a “higher” or “better” plan, but the name “distinguished” belies this point, as does the fact that only students in the distinguished path are eligible for automatic college admission under the Top 10% rule.
So instead of defaulting students into the higher pathway, as current law does, HB 5 contemplates a process whereby every 9th grader, in collaboration with her family and school counselor, elects a pathway in the first year of high school - when some of them will still be 14 years old. For some students with strong parental involvement, I’m sure this will be fine. But for some students - who may be capable of greater things than their individual counselor, their parents, or even they themselves believe - there is a very high risk that this decision will constrain their future opportunities in a profound way.
There should be no stigma attached to families making an informed decision, perhaps halfway through a student’s high school career, to opt out of a college-prep curriculum and seek more career preparatory coursework. I’m not saying that defaulting every kid into a college prep program means all kids want or need college prep coursework. But when we presume all kids are capable of college level work, and start preparing them for it, there is no doubt in my mind we change the futures of some kids whose promise may not otherwise have been recognized.
60% of the students in Texas schools are eligible for free or reduced lunch. It is the fastest growing sector of our student population. If we do not find ways to steer increasing numbers of them toward higher education and knowledge-economy skills, we will pay a high price down the road.
2. HB 5 eliminates all standardized assessment of student achievement beyond freshman and sophomore level coursework, and reduces us down to one assessment per subject area.
As I said before, I believe testing needs to be reformed in Texas - by moving the tests to the end of the school year; by releasing the test for review by parents and educators; by reducing the stakes so they can be used diagnostically rather than punitively - but HB 5’s elimination of all assessments in reading and math beyond the level of English II and Algebra I is a mistake. We know we already have a huge problem with the cost of remedial classes in higher education. By not measuring achievement on a standardized basis beyond Alg I and Eng II, we will have an increasing challenge with students whose transcripts say they are prepared for college level work but who are forced to take costly remedial classes before beginning their higher education. This is a problem that impacts both students and the state adversely.
But I’m even more concerned about the effect of the reduced number of tests on the accountability ratings of campuses serving the most challenging student populations. On those campuses, we have long argued that they should get some recognition not just for the absolute achievement scores of their students, but also for the academic growth they achieve - recognizing that not all students enter high school at the same level. If we measured growth as well as absolute achievement, it would dramatically change the landscape of which schools we think are performing well, and which schools are not.
But under HB 5, we only administer one test in each subject. This makes it impossible to measure growth year-over-year. So a House that claims to want to reduce the burden of high stakes testing has instead concentrated those stakes into a smaller number of tests and focused their effect on the campuses serving our most at-risk students. I don’t get it.
Wisdom from the front lines
As we prepared for HB 5 to come to the floor, I visited with a number of educators who work directly with the standardized tests to find out what they thought. I was surprised at what I heard. I want to share some excerpts from Joy Harris Philpott, Director of Assessment and Accountability at Hays CISD (she’s given me permission to share, and this is reflective of the other opinions I heard):
I feel like the uproar generated by the move to a 4X4 required curriculum and end of course exams in 12 core classes (15 total tests) has resulted in (deliberately or not) the state clamoring for a return to mediocrity. We’ve just graduated the first class on the 4X4 curriculum requirement. We’ve just administered the first round of STAAR end of course tests—yet we’re going to throw a great deal of it out because people feel that we either can’t meet higher standards or don’t want to fund them…
Section 5 of HB5 removes the requirement for a 4X4 curriculum and returns us to three required courses in the core content areas. This will have many students mostly “done” by their senior year in high school, and a “blow off” year during one’s last year of high school does not facilitate a successful transition to college.
…Are we really okay with saying that a student can graduate from high school having only demonstrated proficiency in the above courses (two freshman level, one sophomore level, and one junior level)? In a global economy we’re saying that it’s okay for students to choose to take either world geography or world history and that we won’t assess them on either? While this section of the bill will make my job as district test coordinator easier, it sends the message that we’ve lowered our standards. I’d work twice as hard for higher standards and the resources to support them…
Regardless what the state accountability system ends up being, we’ll be reporting on lowered standards if we remove the 4x4 and say that it’s okay to graduate from high school having demonstrated proficiency in freshman and sophomore level classes.
My final complaint about HB 5 - and this one is admittedly a little unfair, since one bill can’t solve every problem - is that it doesn’t address what should be the most important question of this session: how are we going to improve education in Texas so that our student achievement improves? I have a lot of opinions on this, which I’ll share in another post tomorrow. I’m pretty sure I’ve violated every rule of blogging already with a post this long. Thank god I didn’t deliver it as a floor speech, right?
In the news coverage of yesterday’s oral argument in the Supreme Court, I’ve heard them play this clip of John Roberts saying that advocating for gay marriage - presumably in contrast to a civil union with full legal rights but without calling it marriage - is a debate about a label. And as the attorney (I guess Olson) begins to answer that “labels mean something,” Roberts interrupts:
“If you tell a child that somebody has to be their friend, I suppose you can force the child to say, ‘This is my friend.’ But it changes the definition of what it means to be a friend.”
That seems like exactly the wrong analogy. The correct analogy to this case would be: if two people insist that they very much want to be friends, the state cannot say: no, you can’t be friends. The state could try to define friendship in a way that those two people - who want to be friends - can’t be friends, but it changes the definition of friendship. And in a liberty-impeding, oppressive, unconstitutional way.
Before this legislative session began, Crystal and I decided that it would be my last. By the end of this term I’ll have served 10 years in the Texas House, and it’s been a truly wonderful experience, a great honor, and all that stuff. At the same time it’s been frustrating and at times disheartening, especially since the 2010 elections re-set the balance of power in the Texas Legislature in such a lopsided way.
The norm around here is that you don’t announce that you’re not coming back too early, because lame-duck status can reduce your effectiveness. See, for example, this guy.
So why am I announcing it now? The biggest reason is that there are a bunch of really talented folks interested in running to replace me, and I want them to be able to start introducing themselves to voters without having to tap dance around me.
Plus, I’ve noticed that I’m enjoying this session a lot more knowing that it’s my swan song in the Texas House - I’m enjoying my relationships with the members more, I’m enjoying studying the issues more, and I’m feeling a renewed sense of urgency to make progress on some issues I’ve been working on for several sessions. The expectation is that members will take you less seriously when you’re a lame duck, but I have a sense that it may actually lead to a deepening of the friendships I’ve formed here, which has been one of the most personally gratifying parts of this entire experience.
The floor of the Texas House has to be one of the 100 most interesting places in America. In a space the size of a basketball court, you have 150 people who are, by definition, representative of 150 distinct geographic regions of Texas. I remember being struck when I first got there how you could walk down the aisles of the Texas House and hear every single regional dialect from across the state (and a few from New York.)
If the Texas House didn’t exist for the purpose of democratic self-government, it would need to exist for the purpose of sociological research into what happens when you assemble the most diverse cast of characters imaginable, place them in a small chamber every day for long hours over a five month period, and force them to talk about all the things your grandmother taught you not to talk about in polite company - politics, religion, even sex. All in the context of a reality TV environment in which each action everyone takes has the potential to get them voted off the island. People complain about how unproductive the political system can be; the miracle is that anything ever gets done at all.
As diverse a group as it is, they all have one thing in common: they are all sufficiently sociable to have gotten themselves elected by tens of thousands of voters. There are not many places you can go where everyone greets you with (what at least seems like) genuine enthusiasm, gripping your hand, placing their left hand on your forearm, and looking straight into your eyes as if you are the most important person in the world. My first few days in the House back in 2005 I remember thinking “Holy cow, these are the nicest people in the world.” Then I realized, wait a minute, they’re just using their political mojo on me. I finally figured it out when I started recognizing some of my own BS coming back at me. The members of the Texas House have some pretty good BS.
I know you’re all assuming this means I’m running for mayor of Austin. It doesn’t. I still haven’t decided, and don’t intend to decide until after session is over. I’m very focused on getting the most out of my remaining time as a member of the House. I am thinking about running for mayor, but I’m also thinking about a lot of cool things I could do in the private sector once I’m freed up full time again. I’ve done a lot of work on renewable energy and on education technology, and both are areas where I believe I might have a greater impact through private entrepreneurship than I’m able to have in government.
In the meantime, I welcome all of your thoughts on it. And to the citizens of House District 50, thanks for the privilege of representing you; I’ll continue to do my best for you for the remainder of my term in office.