I made a statistical case last week that wealthy schools, as a class, clearly underachieve on the FCAT in Polk County. This includes charter schools. No one has questioned my data or conclusions. And I’m about to show you that the identical pattern applies in Florida as a whole, when measured on a county-by-county basis.
But first, ask yourself this. Taken at face value, what is the point of the FCAT? What is the point of modern high stakes standardized testing–which I like to call scoreboard education?
In a world of good faith, I think the FCAT is supposed to provide an evaluation tool aimed primarily at teachers and to a lesser degree administrators. It is designed to tell you, based on test score achievement, if your school and teachers are performing well compared to their peers and compared to some abstract standard we establish for them. Are schools and teachers taking the students they have and helping them achieve to their best ability, as defined by test scores?
Thus, one would expect that the ultimate measure of the teacher is the difference between the score a student received and the score he or she would have received if no teacher ever entered the room. That should be the point of the FCAT and standardized testing–to attempt to measure that instructional “value add.”
But that’s just in a world of good faith. We don’t live in that world. Think about that as you look at the charts that follow. Think about the term “value add.” And think about this series of inspiring education quotes.
First, from Maready the Great:
“In review of all the data, magnet, choice and charter schools are making a difference, which should be studied and implemented in other schools. Charter schools are not the total answer but are part of the solution.” — Lakeland Ledger
“Analysis of the data would allow for an open forum to work together in solving the education issues in Polk County. If we do not recognize there is a problem, then there cannot be a solution.” — Lakeland Ledger
Add from loquacious Florida Southern College business professor and McKeel board member Larry Ross:
“We have got in Florida a Republican-controlled Legislature, looking for an alternative to the traditional public education model, which a lot of people think is broken,” he said. “And we have a number of legislators who have been able to sponsor and usher through very favorable charter legislation.” — Lakeland Ledger
And one last person, a powerful state senator named John Thrasher from St. John County. (He is sort of a Townsend family friend, at least of the elder Townsends.) More importantly, he’s an architect of some our state’s education “reforms” over the last decade or so. He’ll tell you himself:
Working alongside Gov. Jeb Bush, we raised K-12 education standards, introduced greater accountability and competition in our schools, and challenged schools to focus resources on the classrooms, not administrations. And just this year, we passed the Student Success Act, which rewards great teachers and empowers charter schools to continue to innovate and improve.
The record and the results are clear. Our reforms have worked wonders in K-12 learning.
Senator Thrasher, if your reforms have worked wonders in K-12 learning, why do your allies on the McKeel board consider the traditional education system “broken?” Is it broken? Or have you worked wonders? Please clarify. And if it’s not broken, why have Republican leaders like you felt it necessary “to sponsor and usher through very favorable charter legislation” designed to upend the traditional system which you supposedly reformed to great success?
With respect, could it be that you are all deeply full of hooeey? That everything you say about education is focus-grouped drivel with no meaning?
“A lot of people” think that. They include me.
I think the evidence suggests that the entirety of Florida’s state-level education bureaucracy and evaluation apparatus serves the true aim of John Thrasher and Seth McKeel’s vision for public education: justifying the creation of free private schools for their kids and grandkids and using your money to do it. They couldn’t do it with vouchers, so they’re doing it with predatory charters and a thousand other cuts. Killing off teacher unions as a viable political opposition is a bonus. But, in the end, it’s about making sure their kids don’t have to share space with riff-raff, with the exception of a few very, very special riff-raffians.
I submit that Florida education policy has nothing whatever to do with “achievement” and quality of educational experience. And as you follow me through the charts and analysis that follow, ask yourself whose perception of the public education debate rings more true: mine, or the muddled incoherence of the underachievers.
Randomness and Order
Check out these two unadorned scatter charts below. Do they show a similar vector in plotting?
You’ve already seen the top chart if you’ve been reading the last couple of weeks. It’s the 2010 3rd grade mean (average) FCAT scores in Polk County, plotted school-by-school against free and reduced population. My analysis of that can be found here if you’re not up to speed.
Concerned that my chosen slice of FCAT world might have been a little fluky, I recently plotted another FCAT-related combination, one as different as possible from Polk’s 3rd grade reading. I chose FCAT 2011, 10th grade Math. Only this time, I did it at a state level, measured county district-by-county district. (This data sample also included a handful of odd, sort of one-off special districts, which make for useful outliers. They show a wide range of possible scores. The spread of possible scores is at least 277 to 371.)
I ask again, do these two charts produce a similar right-to-left flow of dots? If you say yes, don’t you have to ask what accounts for the similarity? I see two possibilities:
1) The random interactions of Polk’s 3rd grade teachers and students in 2010, against all laws of probability, produced a clear statistical pattern relating to free and reduced lunch population. And that pattern matched almost precisely the pattern produced by the random interactions of all of Florida’s 10th grade math teachers and students the next year.
2) There is a rigid, obvious, predictable statistical correlation between free and reduced lunch population and FCAT score–in any combination.
I think the second is more likely.
However, Florida’s entire political education evaluation apparatus and rhetoric operates as if option number one explains the similarity of those charts. Mull that for a second as I note a couple of intriguing data elements not directly related to score.
Florida’s 10th grade Math population skews significantly wealthier than Polk’s 3rd grade reading. That moves its dots way to the right of the state chart’s dots. Polk’s 10th grade Math population is 57 percent, much lower that the 72 percent population in 3rd grade. That suggests to me that we’re either losing a significant portion of poorer kids to some sort of attrition–dropping out or just disappearing–or that fewer eligible families claim free and reduced lunch in 10th grade than in 3rd. Most likely, it’s some combination of both.
Also, both charts produce a very odd gulch between free and reduced populations, at a similar wealth point. For Polk 2010 3rd grade Reading, there are no schools with FRL population between 55 and 48 percent. For the state 2011 10th grade Math, there are no districts between 66 and 61. In Polk, 2/3 of schools are on the poor side of the gulch. In the state, 2/3 are on the wealthier side. I don’t know what to make of it, exactly, but I find it intriguing. It says something about that way people–and socioeconomic forces–distribute data. I just don’t know what it says.
McKeel Empire, Meet St. Johns County
Now, look at the two charts below. They are the same two charts, annotated to reveal some of the identities of the blue dots from above and drawn with a median score line.
As a refresher for non-STEMMY types like me, “mean” equals average. You add up all the scores and divide the total by number of scores to calculate the average that any individual kid would score. The dots are average scores of all students who took the test.
Median is slightly different. Median refers to the score that is dead in the middle of all scores. An equal number of scores are higher and lower. So I’ve drawn a median line for mean scores. This smooths out the effect of outliers; and I think it makes better sense for illustrating underperformance and overperformance. If there’s an automatic way to draw a median line on my Excel chart, I couldn’t figure it out. So I did it manually. My line could be slightly off, but any tiny tweaks do not remotely change the pattern. Please draw your own line if you don’t like mine.
As you can see, wealthier kids and their teachers underperform on the FCAT when the obvious, predictable correlation of mean score and FRL population is included. Said again, poorer traditional schools clearly get more out of their kids–in terms of test scores–than do fancy schools. In Polk County and in Florida. Just look at the dots and the lines. Who is under the median line? Who is above?
Clearly, we have a crisis in rich kid education. Is little Ethan learning? Not really. At least not compared to the kids his parents won’t let him play with.
Let’s look again at the state chart:
You will notice that the champion underperformer in the state is its richest county: John Thrasher’s St. Johns County. Congratulations on your legacy of reform, senator. The greatest overacheiver, rural little Gilchrist County, scored a mean of 339 with a 59 percent FRL. St. Johns, with an astonishing 17 percent countywide FRL, managed a 344. And I’m sure the usual suspects crowed about it.
The only thing I know about Gilchrist County, which is northwest of Gainesville, is that it contains a little town called Trenton. My Palatka American Legion baseball team got into a absolutely awesome bench-clearing brawl there in the summer of 1988. Players, parents–it was a spectacle. I was warming up in the bullpen at the time. I am pretty slow on a good day, and that wasn’t one. So I managed not to get to the pile in time to throw a punch. Good times. Little did I know that players and fans with which we traded body slams and twangy barbs would be producing Florida’s best scholars in the 2000s.
Today, I feel certain that Gilchrist County is Florida’s fastest growing county because we all know that test score achievement–not demographics–sell neighborhoods and regions. Right? Maybe some economic development professional con confirm that for me.
Polk County comes in almost exactly on the median line, which, I have to say, feels right. We’re not overperforming; we’re not underperforming, as a county. But look at our school-by-school chart.
Which schools are keeping us from overperforming? The McKeels, Valleyviews, Lincolns, and Ridgeviews. With the exception of Lakeland Montessori (usual disclosure: my son goes there), rich schools in Polk — and especially predatory charters — are not pulling their own weight. If they were, Polk’s numbers would clearly exceed the median line. I suspect it’s the same everywhere else.
Here’s the key question for most wealthy schools: a handful of schools show that wealthy student populations similar to yours are capable of scoring of much higher than your populations are scoring. Why aren’t your kids scoring that high? What are you doing wrong?
In fairness, one can rightly argue that school-by-school 2010 3rd grade FCAT Reading in Polk is not the same pot of data as county-by-county 2011 10th grade FCAT Math. I agree. But, really, does anyone think any other combination is going to look any different on a scatter plot? Really? If I were Mitt Romney, I’d bet you $10,000. However, I am not Mitt Romney. So I’m just going to post at least one additional combination a week, in perpetuity, or until I get tired of it. And we’ll see just how consistently this pattern of right-to-left upward angles reproduces itself.
What are you good at? What’s your value add?
To close, let’s refer back to the question I posed at the beginning.
What is the point of the FCAT? What is the point of modern high stakes standardized testing–which I like to call scoreboard education?
My answer: In a good faith world, it is designed to tell you, based on test score achievement, if your school and teachers are performing well compared to their peers–and compared to some abstract standard we establish for them.
What do you think? Is it doing that? Does our evaluation tool work for anyone? It seems to be harming rich kids with a false sense of achievement. And it seems to harm poor kids by giving McKeelish types the PR ammunition to slander the overperforming teachers at traditional schools and plunder their resources and generally drive them out of education.
In my view, Florida’s scoreboard education apparatus is nothing more than an elaborate and opaque Census, telling us what we already know about where wealth is distributed. And don’t you conservatives hate the Census? Why do you want to duplicate it on state level?
I, for one, would chuck all of this garbage out tomorrow. There are a billion more productive ways to pursue accountability if we care at all about making that abstract concept real. Our current system employs none of them because it is not interested in evaluation. It is interested in justification and rationalization of predatory policies. If we trust that our teachers and administrators are not fundamentally failing, much of the scoreboard education agenda falls apart from its own incoherence. And if you don’t think McKeel and St. Johns County are failing, then no one else is either.
I also asked you, dear reader, to think about the term “value add.” What is the value add of predatory charters, based on what I’ve shown you? What is the McDump Empire good at?
I would argue that it’s only good at screening and getting rid of kids. And the numbers, quite frankly, show it’s not good enough at that. I do, however, have great faith in its commitment to improving selectivity, at least. That’s its product. Larry Ross very helpfully said it last week.
He knows McKeel’s target audience–people who want to say they send their kids to an elite private school without paying for it. Trust me, the McKeel folks believe even more strongly than I do in the rigid predictable correlation of FRL and raw test score. They count on it. They know they can use the wealth and demographics of their student body to expand their government-funded business. They know that achievement is irrelevant to marketing.
As my late lamented economics professor grandfather used to say, “Isn’t education a wonderful process?”