In Which I Get Debunked at Fivethirtyeight and Set Off A Minor Media Firestorm

Well, well. So my post comparing HDI scores of US states to foreign countries has gone a bit viral. It got picked up by Catherine Rampell at Economix, a New York Times blog; then by Richard Florida, of all people, posting at Andrew Sullivan's blog; and then a bunch of other places. I have to say, though, most folks seem less interested in my insightful analysis that Mississippi is kinda like Albania than in the Wikipedia map I used for the post:



Well, so Andrew Gelman, posting at fivethirtyeight.com, got ahold of the map, and it seems to have irked him. Says Gelman:

Is Alaska really so developed as all that? And whassup with D.C., which, according to the table, is #4, behind only Connecticut, Massachusetts, and New Jersey? I know about gentrification and all that, but can D.C. really be #4 on any Human Development Index worth its name?

Time to look behind the numbers.

Gelman goes on to bring his statistical chops to bear on the data, which came from this table from Wikipedia, and his analysis is frankly a bit beyond my pay grade. The upshot is, he's an expert, and he's dubious about the data. So I decided to look into the matter a bit more.

The Wikipedia page that lists HDI by state claims that the data come from the American Human Development Project. So I contacted them to find out if the numbers from the Wikipedia page were reliable. Kristen Lewis, a Co-Director of the project, emphasized that the AHDP's HDI index - though calaculated based on measurements of education, health, and income, just like the UN's index - uses, as she put it, "different indicators that serve as more reliable and meaningful proxies in the U.S. context." She went on to say:

To avoid creating the impression that our index was comparable to the UNDP global HDI published every year that ranks all the world's countries, we used a different scale. Rather than 0 to 1, we used 1 - 10. In addition, we said in several places that our index was not directly comparable to the UN index.

We still wanted to make international comparisons, but we did so in our book by using more discrete indicators. So for instance, we compared our incarceration rate to those of other countries, noting that ours was higher than those of China or Russia; we compared our infant mortality rate, noting that in parts of Mississippi, the infant death rate was on par with those of Libya and Thailand, etc.

I have no idea who created that table in Wikepedia and what methodology they used to convert our scale to the UN scale. We have data tables on our website where the person could have gotten the LIEX by state; I'm not sure what he or she used for income, but if they used median personal earnings, it's not comparable to the UN scale and if they used state GDP, they would run into the problems described above; and in terms of education, he or she may well have used school enrollment, which we have, but I don't know what they would combine it with as we don't have literacy by state and, again, the educational attainment figures would not be comparable.

So there you have it. Who knows where the numbers on which the above map is based came from? It seems, to the extent that anyone drew their data from the AHDP report, their methodology must necessarily have been flawed, since the AHDP data are incompatible with the data the UN uses for their HDI, and the table used for the map above used values based on the UN's scale.

However, the AHDP does have its own maps of their data, to wit:



Note that the HDI presented here is, as Lewis points out, on a 1-10 scale, rather than the UN's 0-1 scale. Note also that this map closely mirrors the Wikipedia map; indeed, the ordering of the states is nearly identical. But the ratios of HDI between the states are not. For instance, look at the bottom 15 or so states on the AHDP list (pdf):

36. Missouri - 4.54
36. Nevada - 4.54
38. South Dakota - 4.53
38. Wyoming - 4.53
40. New Mexico - 4.49
41. Idaho - 4.37
42. Montana - 4.34
43. South Carolina - 4.27
44. Kentucky - 4.12
45. Tennessee - 4.10
46. Oklahoma - 4.02
47. Alabama - 3.98
48. Arkansas - 3.86
49. Louisiana - 3.85
50. West Virginia - 3.84
51. Mississippi - 3.58

There does seem to be something of a "long tail" - several states seem to be pretty significant outliers from the national median. But the conclusion I had drawn based on the Wikipedia table was that there was a core of eight states - the bottom eight on this list - that were relatively close to each other on the development scale but which were collectively far below not just the other states in the US, but just about anywhere else in the developed world. But based on the AHDP table, there is not such a clear break - more of a gentle downward slope as you get towards the tail end of the distribution. And while it may or may not be the case that the level of development of these states are well outside the mainstream of other developed countries, there's no way to tell based on this data alone.

For Gelman's part, he tongue-in-cheekily proposes another metric for levels of development of US states:



Gelman reflects:

Why do I have such strong feelings about this? It's probably a simple case of envy, that this little bit of index-averaging has probably received more publicity than all of my life's research put together, envy that it has received so much funding. I'm sure they all have had good intentions, but I think something went wrong, at least with this part of the project.

But maybe I'm thinking about this all wrong: these folks are clearly doing well, so maybe I should emulate them. I'll start by making maps of everything ranked by state, and we'll see how that goes.

To that I can only say: map away, sir!

So, to sum up, I have a couple points. First: it is really danged difficult to find a measure of HDI by US state that can be compared to other countries. But it's such an inherently interesting - even important - question. The US is a huge country, diverse in every way. To really understand our place in the world, we need more fine-grained data than national scale measurements provide. That's what makes the AHDP cool. And it's nice to be able to make intra-US comparisons between states. But it would be fascinating to be able to compare states, in as close to an apples-to-apples way as possible, to other countries. It was curiosity about such comparisons that led me to write the original post, and it was probably a similar sort of curiosity that led whoever drew up the map on Wikipedia to do so. But in all the vast, vast internets, I can find no such comparisons. So, social scientists, what's the hold-up? Is West Virginia more developed than Serbia? The people want to know!

Second: good lord, but things do get a life of their own on the internets, don't they? Please note that the American Human Development Project has done some really nice work. I mean, their report has a foreword by Amartya Sen, for crissakes! The greenish map above is not their responsibility, but the brownish one IS theirs. And I encourage everyone to go check it out at their site; there is a ton of interesting information there.

"Paper Cuts": The Death of the Newspaper Industry

Paper Cuts is tracking the ongoing labor carnage in the newspaper industry in the US. They've got an interactive map with details on newsroom job losses around the country.



The South Florida Sun Sentinel. The Rockingham News. The Washington Post. The Pocono Record. The Atlanta Journal-Constitution. These are only some of the papers that have announced layoffs just since Saturday. The combination of the wan economy and the outmoded business model of newspapers has meant disaster for this industry. It looks increasingly like US newspapers in their present form may not be long for this world.

But is that such a bad thing? It would certainly be a terrible thing if journalism didn't survive. But there's surely no chance of that happening - there will always be a demand for journalism. In the future, though, it's likely to be hosted by media other than cheap, inky paper delivered in a bundle to your doorstep. The web is the obvious new venue for journalistic content, and that medium's been growing for years now, of course. But the real interesting test for the future of journalism will be what happens when a major city finds itself without any major newspaper. What will fill the void? My guess is: a proliferation of publications, especially online but in print as well, focused on much narrower niches. This, of course, has been the trend on the internet - the proliferation of micro-focused content venues (ahem) with little or no overheard costs, often motivated by individuals' passions rather than profit. The upshot has been that there's an incredibly richer array of content available to consumers now than there was ten years ago, even as newspapers have declined. (I've just been surfing around for information on Houston's light rail plans, for instance, and I've found way more information than I ever could have by simply waiting around for the paper to show up every day.)

This is the way media content is produced now, and in this context newspapers (like network television) are an anomaly: they offer breadth, not depth, as they rely on serving the interests of a mass audience. But there's nothing especially natural about combining news with sports scores, book reviews, classified ads, real estate listings, comics and all the rest of it, like a typical big-city paper does. The only reason to do so is if the medium that is best suited to distribute lots of information quickly and cheaply does not allow the consumer to exert selective control over content - then an editor has to decide what will appeal to the greatest audience, and deliver it all together in a big bundle. That was the best system for quick and thorough information dispersal during the 20th Century, but it no longer is.

The fact that this anachronistic and quirky medium is unlikely to last much longer in its present form doesn't, as I said, mean the end of journalism. But the questions are, over the long term: will journalism (of the 'serious' investigative sort) become a non-profit enterprise? And are we likely to suffer in the short-term from a lack of established journalistic institutions to act as a watchdog against corporations and government? But of course, for everyone in the industry who are currently suffering from the turbulence of economic and technological change, the questions are much more immediate and personal.

UPDATE: For an interesting and much more comprehensive expression of this general line of thinking, see Steven Berlin Johnson.