Thursday, May 21, 2009

In Which I Get Debunked at Fivethirtyeight and Set Off A Minor Media Firestorm

Well, well. So my post comparing HDI scores of US states to foreign countries has gone a bit viral. It got picked up by Catherine Rampell at Economix, a New York Times blog; then by Richard Florida, of all people, posting at Andrew Sullivan's blog; and then a bunch of other places. I have to say, though, most folks seem less interested in my insightful analysis that Mississippi is kinda like Albania than in the Wikipedia map I used for the post:



Well, so Andrew Gelman, posting at fivethirtyeight.com, got ahold of the map, and it seems to have irked him. Says Gelman:
Is Alaska really so developed as all that? And whassup with D.C., which, according to the table, is #4, behind only Connecticut, Massachusetts, and New Jersey? I know about gentrification and all that, but can D.C. really be #4 on any Human Development Index worth its name?

Time to look behind the numbers.
Gelman goes on to bring his statistical chops to bear on the data, which came from this table from Wikipedia, and his analysis is frankly a bit beyond my pay grade. The upshot is, he's an expert, and he's dubious about the data. So I decided to look into the matter a bit more.

The Wikipedia page that lists HDI by state claims that the data come from the American Human Development Project. So I contacted them to find out if the numbers from the Wikipedia page were reliable. Kristen Lewis, a Co-Director of the project, emphasized that the AHDP's HDI index - though calaculated based on measurements of education, health, and income, just like the UN's index - uses, as she put it, "different indicators that serve as more reliable and meaningful proxies in the U.S. context." She went on to say:
To avoid creating the impression that our index was comparable to the UNDP global HDI published every year that ranks all the world's countries, we used a different scale. Rather than 0 to 1, we used 1 - 10. In addition, we said in several places that our index was not directly comparable to the UN index.

We still wanted to make international comparisons, but we did so in our book by using more discrete indicators. So for instance, we compared our incarceration rate to those of other countries, noting that ours was higher than those of China or Russia; we compared our infant mortality rate, noting that in parts of Mississippi, the infant death rate was on par with those of Libya and Thailand, etc.

I have no idea who created that table in Wikepedia and what methodology they used to convert our scale to the UN scale. We have data tables on our website where the person could have gotten the LIEX by state; I'm not sure what he or she used for income, but if they used median personal earnings, it's not comparable to the UN scale and if they used state GDP, they would run into the problems described above; and in terms of education, he or she may well have used school enrollment, which we have, but I don't know what they would combine it with as we don't have literacy by state and, again, the educational attainment figures would not be comparable.
So there you have it. Who knows where the numbers on which the above map is based came from? It seems, to the extent that anyone drew their data from the AHDP report, their methodology must necessarily have been flawed, since the AHDP data are incompatible with the data the UN uses for their HDI, and the table used for the map above used values based on the UN's scale.

However, the AHDP does have its own maps of their data, to wit:



Note that the HDI presented here is, as Lewis points out, on a 1-10 scale, rather than the UN's 0-1 scale. Note also that this map closely mirrors the Wikipedia map; indeed, the ordering of the states is nearly identical. But the ratios of HDI between the states are not. For instance, look at the bottom 15 or so states on the AHDP list (pdf):

36. Missouri - 4.54
36. Nevada - 4.54
38. South Dakota - 4.53
38. Wyoming - 4.53
40. New Mexico - 4.49
41. Idaho - 4.37
42. Montana - 4.34
43. South Carolina - 4.27
44. Kentucky - 4.12
45. Tennessee - 4.10
46. Oklahoma - 4.02
47. Alabama - 3.98
48. Arkansas - 3.86
49. Louisiana - 3.85
50. West Virginia - 3.84
51. Mississippi - 3.58

There does seem to be something of a "long tail" - several states seem to be pretty significant outliers from the national median. But the conclusion I had drawn based on the Wikipedia table was that there was a core of eight states - the bottom eight on this list - that were relatively close to each other on the development scale but which were collectively far below not just the other states in the US, but just about anywhere else in the developed world. But based on the AHDP table, there is not such a clear break - more of a gentle downward slope as you get towards the tail end of the distribution. And while it may or may not be the case that the level of development of these states are well outside the mainstream of other developed countries, there's no way to tell based on this data alone.

For Gelman's part, he tongue-in-cheekily proposes another metric for levels of development of US states:



Gelman reflects:
Why do I have such strong feelings about this? It's probably a simple case of envy, that this little bit of index-averaging has probably received more publicity than all of my life's research put together, envy that it has received so much funding. I'm sure they all have had good intentions, but I think something went wrong, at least with this part of the project.

But maybe I'm thinking about this all wrong: these folks are clearly doing well, so maybe I should emulate them. I'll start by making maps of everything ranked by state, and we'll see how that goes.
To that I can only say: map away, sir!

So, to sum up, I have a couple points. First: it is really danged difficult to find a measure of HDI by US state that can be compared to other countries. But it's such an inherently interesting - even important - question. The US is a huge country, diverse in every way. To really understand our place in the world, we need more fine-grained data than national scale measurements provide. That's what makes the AHDP cool. And it's nice to be able to make intra-US comparisons between states. But it would be fascinating to be able to compare states, in as close to an apples-to-apples way as possible, to other countries. It was curiosity about such comparisons that led me to write the original post, and it was probably a similar sort of curiosity that led whoever drew up the map on Wikipedia to do so. But in all the vast, vast internets, I can find no such comparisons. So, social scientists, what's the hold-up? Is West Virginia more developed than Serbia? The people want to know!

Second: good lord, but things do get a life of their own on the internets, don't they? Please note that the American Human Development Project has done some really nice work. I mean, their report has a foreword by Amartya Sen, for crissakes! The greenish map above is not their responsibility, but the brownish one IS theirs. And I encourage everyone to go check it out at their site; there is a ton of interesting information there.

63 comments:

Mark A. Sadowski said...

Chachy,
Sorry to bust your balls (as a commenter on multiple sites) but I thought it was important to point out that the Wikipedia data and map were questionable and that the US HDI was not comparable to the UN HDI.

On your subsequent points, although it is difficult to find a US HDI that is comparable, it is not that difficult to create one. The data is all out there, one merely needs to compute it. The UN HDI lists its methods and I believe that all the data is out there: state GDP per capita (at the BEA), Life expectancy (at the CDC), and adult literacy and educational enrollment (where at this particular moment I am not sure, but I'm confident it is out there).

Now, the question is, who is up to the task of replicating the UN HDI's work on individual US states? Moreover, who is up to the task of editing the numerous incorrect Wikipedia entries?

T. J. Hairball said...

Well, I *thought* this would be easy, but it turns out that traditional literacy rate data by state is difficult to obtain.

The NCES collects data related to literacy by state, but it does not appear to include a traditional literacy rate. Enrollment (and enrollment-age population) are all available on the census website, of course, with a bit of work to put them together, but the US census no longer covers literacy, which is a more major part of the index.

Chachy said...

No worries, Mark. I post in a spirit of "eh, let's suss this out a bit..." so any constructive criticism is appreciated. (It's just rarely this sort of high-profile media adventure...)

It would seem, in principle, to be a pretty straightforward thing to compile UN HDI numbers for the states. Which is why I'm surprised it (apparently) hasn't been done. I'd still be really interested to see such numbers...

TJ - appreciate the effort. Another problem might be defining literacy. There are a number of ways to do so, and I'm not sure which one would be compatible with the UN methodology, if any.

Mark A. Sadowski said...

Chachy,
Actually, on reflection, as a concept, your site rocks!

I fiddled today with the problems associated with constructing an UN HDI consistent index for the US satets today and I observed the following:

1) The income component is easy. Just take the real US GDP per capita statistics from BEA and multiply them by the US GDP deflator for 2005 (approximately 1.1306).
2) The health component is similarly easy. The CDC has the state level statistics but so does the US HDI website. They are essentially consistent (small deviations).
3) T.J. Hair is right. The education component is a bear.
a) First of all, the literacy rate. The last time the Census collected state level literacy statistics is 1970. The last time they collected it on a national level is 1979. Why? Because literacy was essentially universal by the 1970's in the United States. For the UN HDI, any nation that has literacy rates above 99% or that does not collect such stats is alloted a score of 0.99 for that component. If one looks at the 1970 state level stats you will observe that the lowest literacy rate was for Louisiana or 97.2%. Based on even the lowest rate of decreases in the rate of literacy it is clear that by 1990 all the states in the United States probably would have had a literacy rate of 99% or higher by the UN's low standards.
b) It turns out that the combined educational enrollment rate may be the thornier problem. The American HDI website lists such data but it is not consistent with the data reported in the UN HDI report. It is lower by a factor of 0.93. I suspect that the problem is not with the numerator (total enrollment) but with the numerator (population in relevant age group). In any case I rescaled the data by a factor of 1.075.

The results? Missippi scores at 0.901. Connecticut scores at 0.969. Hawaii scored the highest at 0.974.

What did I learn from this exercise? The biggest differences in HDI by the UN standards occurred because of the differences in longevity. Twenty of the US states max out on income. All perform well by the low educational standards of the UN index. In longevity however the US does not perform very well.

On reflection, what I think we need is not an index that is consistent with the UN as it is designed for comparisons between developing nations or even an index based on US standards. What I think is needed is an index that is designed to compare the US states to the EU states and the more advanced states of East Asia etc.

I would recommend GDP per hour worked for the income component (OECD) as a measure of productivity, life expectancy for the health component (widely available), and a measure of educational attainment similar to the US HDI index. This would show how well the different US states fare compared to the rest of the advanced world in terms of things that really differentiate at this level of human development attainment.

I honestly think this is a good project for someone so inclined (maybe even myself).

Chachy said...

Interesting stuff. If Mississippi is really .901 on the UN scale, that would ba about comparable to Portugal or Bahrain or Qatar, and slightly better than the Czech Republic and Hungary. That actually seems pretty respectable.

In any case, let me know if you come up with a set of mappable data.

E. Martins said...

First, many thanks to Cachy for taking the time to check with us at the American Human Development Project on the validity of the state HDI data that was being circulated. That cleared a lot of the confusion that was spreading around in blogland!

Now, on to the more recent comments…

As Mark and T.J. quickly found out, it is not as easy to create an HDI for US states comparable to the UN HDI. Adult literacy data is the main difficulty, it just isn’t available (and believe me, we looked everywhere for it). And, while you can easily find states’ GDP per capita, it really isn’t a very good measure of the income appropriated by the local population, since states are much more open economies than countries, and substantial portions of the income generated within the community are used to remunerate production factors owned by persons who do not reside in that community.

Still, even if we assumed this data would be readily available and relevant, would such an index be really that interesting?

To create the three HDI components, a process of normalization is used, so that each component will range from 0 to 1. For the income component, a minimum value of $100 per capita (in PPP terms, but that is not relevant here) is used, and a maximum value of $40,000 per capita. The log of GDP per capita is taken, and the Income index is calculated. Countries with a GDP per capita greater than S40,000 are assigned an income index equal to 1, the maximum possible value. In 2007, 16 US states had GDP per capita greater than $40,000, and would have an income index of 1. The lowest GDP per capita (Mississippi) was $28,541, resulting in an income index of 0.94. Thus, all states would be clustered in the upper end of the income index range.

In the education component, the clustering would be even more noticeable on the literacy index, given the almost universal adult literacy in the US. Virtually all the variation on the education component would be due to the enrollment rates, and even this would be dampened by mandatory education.

So, all that would be left is life expectancy. A US HDI index calculated using the standard UN methodology would basically amount to an index based on life expectancy, which shows some (but not that much) variation within states – D.C. would have the lowest index, at 0.81.

As noted by Mark, this would not capture the variations in income, health and education outcomes within the US. It is precisely for this reason that we adapted the UN methodology, so that the resulting index would be more suitable in the context of an industrialized nation. The methodology is described in detail on the book, “The Measure of America”, and will also be posted on our site. Using the adapted methodology, we computed a Human Development Index not only for the 50 US states and DC, but also for all 436 congressional districts, and we are now applying it to counties and groups of counties.

In a nutshell, a US HDI using the standard UN methodology would place virtually all US states at the very top of any list, and would show almost no variation within states. The modified American Human Development Index provides a much more nuanced picture of human development in the US, and is a much more useful tool for analysts and policy makers.


─ Eduardo Martins, co-author of “The Measure of America: American Human Development Report 2008-2009″, and Statistics Director, American Human Development Project.

Chachy said...

EM - thanks for the comment! To be honest, it surprises me a little bit that a state like Mississippi would rank "near the top" of a measure of HDI based on the UN method compared to other countries. Would it really rank above other countries in Western Europe?

Anyways, I look forward to the county data.

Mark A. Sadowski said...

Chachy and Eduardo Martins,
I wasted a considerable amount of this day trying to create US scores based on the UN HDI. It is as pointed out by Eduardo not terribly interesting as it ends up being mostly a measurement of health. I would be happy to share my estimates and methods if you were still genuinely interested.

On a more important note I created my own scale largely on the principles that I suggested previously (using OECD ranges as a guide). I intentionally meant it to be challenging to the US states. I used life expectancy at birth for health, combined educational enrollment for education, and an estimate of GDP per hour for income. By this measure the US measures between the UK and Canada on average and Texas is near to this average. Connecticut performs best of all, between France and Australia. Mississippi performed worst of all, between the Czech Republic and Portugal. Again I would be happy to share my estimates and methods if you were genuinely interested.

Chachy said...

Mark - Sure, if you want to send me your data I'd be happy to do a post on it.

crazyloko said...

数据采集

123 123 said...

Cool article you got here. I'd like to read more about that matter. Thank you for posting this data.
Joan Stepsen
Tech gadget

Dieran Blog said...

Hi...Your post really got me thinking man..... an intelligent piece ,I must say.

liposculpture said...

thanks for sharing useful information, you are indeed a good person ...

home improvement|Last Health News

student loan said...

This is a really good read for me, Must admit that you are one of the best bloggers I ever saw.Thanks for posting this informative article.

Contextual links|home improvement

finance information said...

Very informative, definitely worth reading. Thank you for such great post ^^


world flash game|Watch Movie Online 90210

Contextual links said...

Your blog is very much good. I am very much impressed by your blog content ........

student loan|home improvement

Finance Tips And Reference said...

Thanks for the article, this site really enriches me ...



gadgets and tech tips|business review

Entertainment Blog said...

your wesbite good, I enjoyed visiting here ....




dunia belajar

Home Management said...

I think this type of article is useful for many readers blog.Thank you so much.


Beasiswa s1

Company the Film said...

Can only say thank you for this information ...




Company the Film

coklat said...

This is one of the best websites I have ever visited, and I certainly would leave a trail with the comment that "YOUR WEBSITE VERY GOOD"

Anonymous said...

yesterday, my friend told me that there is a great site.,. and I tried to visit the site on purpose .... and he was right, this site is really good .... tomorrow I'll do the visit here again, hopefully there is new information that I get on this site ..




Bluetooth Headphones

Automotive said...

Visiting this site is fun, because I can get new information to add knowledge ... thanks for the owner and the author of this site ....

information ,technology, business,autos, entertainment

Anonymous said...

A beautiful morning to start browsing activity, and I was lucky enough to visit this site ....



Soccer Shoes World Cup Merchandise

Sport Watches said...

Only one word can I say, it is "GOOD"

Drafting Table

imagetoon said...

Useful information, I know many things after reading this article, because it gives you inspiration ...

acne solution said...

thank you for presenting the news that I needed ,....

acne solution said...

thank you for presenting the news that I needed ,....

tn requin said...

sleeping bag
tablecloth
travel luggage
car gps
car parts
GPS navigation
portable GPS
lcd tv
flat tv
digital picture frames
unlocked cell phones
wedding dresses
wedding jewelry
chaussures mode
chaussures Femmes
women's shoes
round tablecloth
discount handbags
chaussures sport
chaussures pas cher
fashion bedding
id lanyard
lanyard
creative gadgets
gift ideas
kids wall stickers
wall stickers
bath curtains
kitchen faucet
bathroom faucet

nadja said...

Wonderful blog! I truly love how it? s easy on my eyes as well as the data are well written. I am wondering how I can be notified whenever a new post has been made. I have subscribed to your rss feed which need to do the trick! Have a nice day!
broadband service
Android phone
Link Building Service
Compare Broadband
Beautiful Baby Gift
Beautiful Baby Stuff
Cheap Travel
Cheap Hotel
Best Racing Car
Top banner
best music

MesotheliomaHelp said...

Thanks for your article,

Best E Commerce Software
iPhone alarm glitch
Mobile Application Development
Mobile Software
Mesothelioma Help

Asbestos Cancer Treatment

Mesothelioma Attorneys

kiara said...

Costa Rica Tours
Tours Costa Rica

berita terbaru said...

Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon
point blank
Biro Jodoh Online Gratis
Ramalan Bintang Zodiak Terbaru
Ramalan Shio Zodiak Tionghoa
Kumpulan Contoh Naskah Pidato
Cara daftar point blank
Point Blank Online Indonesia
Kata-kata Bijak dan Motivasi
Koleksi Status Keren Facebook
mobile phone
my cupu
Harga Spesifikasi HP
Kumpulan berita terbaru

Height Increasing Tips - Get Your Optimum Results said...

great article..

How To Get A Taller - Get You Optimum Results in Weeks
Exercises To Get Taller – Grow Taller Fast
Natural Ways To Get Taller - 100% Naturally
How To Look Tall - Tips On How You Can look Tall 2-3 inches Now!

Penis Enlargement Pills said...

I am thoroughly convinced in this said post. I am currently searching for ways in which I could enhance my knowledge in this said topic you have posted here. It does help me a lot knowing that you have shared this information here freely. I love the way the people here interact and shared their opinions too. I would love to track your future posts pertaining to the said topic we are able to read.

susu colostrum said...

In a nutshell, a US HDI using the standard UN methodology would place virtually all US states at the very top of any list, and would show almost no variation within states. The modified American Human Development Index provides a much more nuanced picture of human development in the US, and is a much more useful tool for analysts and policy makers.

Konsultan Marketing-Konsultan Perusahaan said...

thanks for sharing

Parcel natal
Rumah Sakit Mitra Kemayorans
belajar internet marketing
how to prevent premature ejaculation

admin virus said...

thanks for sharing useful information, you are indeed a good person ...

kudhis said...

nice today...thanks

Profile said...

,FB,
,FB,
,Harga Handphone Terbaru,
,Jadwal Liga Inggris,
,Klasemen Liga Inggris,
,Point Blank,
,Ramalan Jodoh,
,SMS Cinta,
,Zodiak Hari Ini,

justin bieber said...

A really usefull post - literacy rate data by state is difficult to obtain. A big thank you I wish you will not mind me writting about this article on my blog I will also link back to this post Thanks

nisioi_cosmin said...

There are actually plenty of particulars like that to take into consideration. That may be a nice point to convey up. I provide the ideas above as common inspiration but clearly there are questions like the one you bring up where an important thing can be working in honest good faith. I don?t know if best practices have emerged round things like that, however I am positive that your job is clearly identified as a good game. Each boys and girls really feel the affect of just a second’s pleasure, for the remainder of their lives.
Remixuri 2011 | Muzica 2011

renegade said...

baby and mother health online

main leadeship conference

Look Ok, pretty and great

raise your fund

Copper and Still industries

pick a bill software

art of wellness

just rate food and beverage

Indonesia Statu Shop

Indonesia Statu Shop

the art of music

Buy green furniture product

banner stock

Clarington Bed and breakfast

daniel rivas music

EM Technologies group

local surfer

Pacific travel

Ponyx Fashion shopping
On your subsequent points, although it is difficult to find a US HDI that is comparable, it is not that difficult to create one. The data is all out there, one merely needs to compute it. The UN HDI lists its methods and I believe that all the data is out there: state GDP per capita (at th
Passion to tour and travel

recycling technology

make health and wealth

Finance and Insurance Companies' show

Sea Product Export

A.J. Jacksoon said...

Hit upon away as of home how with the purpose of get amazing done a switch cell phone reverse phone lookup or else a cell phone search before yet a cellular phone number lookup lookup with the aim of come across away as of home who is bringing up the rear with the purpose of being with the aim of has been calling you this complete point in time ok.

Anonymous said...

Invalidate cellular phone search pages akin to this lone spirit give you all kinds of telephone lookup information so so what time to you can ultimately find not by the side of home their name, address, where they be, reverse phone lookup their current age along with to a excellent extent extra. So as soon as you necessitate to run through a cellular phone number lookup today, only perform clear in your mind to splurge this effective tool with the intention of come across away on or after home all kinds of reorder mobile phone search particulars down that guise today.

Anonymous said...

I enjoy this nice change cellular phone search or cell phone search otherwise telephone number search spot. I be able to gather greatly of cell phone lookup things a propos reverse cell phone search here these days as soon as I checkered away this guide here.

sam witular said...

Nice be here blogwalking
surat lamaran kerja, facebook login, status lucu, zodiak hari ini, zodiak, ramalan jodoh, ramalan bintang, mivo tv, komik naruto, daftar facebook, daftar twitter, aplikasi blackberry, harga blackberry, cheat gta san andreas, pidato, kata bijak, edit foto online, video lucu, tv online

sam witular said...

fb, media online, e991, kue kering, smadav, kata motivasi, kata cinta, kata indah, puisi cinta, puisi persahabatan, kata mutiara cinta, sms lucu, sms cinta, galaxy mini, cinema3satu, idm, internet download manager, antivirus terbaik, google terjemah

betting bonuses said...

I do agree with all of the ideas you’ve presented in your post. They are bet365 really convincing and will definitely work. Thanks, betting odds - I really don't understand a thing what was told around here. uk betting tips All I can see are just bunch of spam comments. Grr. I guess discussions about the topic here is not possible anymore. Thanks, ladbrokes

album foto said...

I think people fear idea management and innovation because there's a significant number of ideas that simply won't plan out.......Nice statement.....keep posting

asuransi pendidikan said...

this day I've been searching for information on various issues, this I found very good and I would like to congratulate you for your work.

design interior jakarta said...

I think that this post is one of the best that i have read in my life, congrats you did a great job,.

perusahaan security said...

the difference between the content spammers and most corporate innovators is that the smaller

jual wine said...

I'm writing to you because I just came across a business that I think has great potential. It lets you save money on almost everything. Make money from almost everything,

coklat valentine said...

If growth is important to a firm, and if growth is dependent on offering existing products and services to new customers

belajar internet marketing said...

innovation is very natural and happens in the "real world" as new plants and animals colonize new ecological niches.

konsultan perusahaan said...

I wonder how you got so good. This is really a fascinating blog, lots of stuff that I can get into. One thing I just want to say is that your Blog is so perfect

kuliner bandung said...

We generally think most specifically about the risk associated with a new product introduction

how to stay erect said...

I would be aware that as somebody who really doesn’t comment to blogs a lot (in actual fact, this may be my first put up), I don’t think the time period “lurker” is very flattering to a non-posting reader.

viagra online pharmacy said...

Congratulation that you reached great sites and media. I have to admit that it is quite easy to understand.

Hair Transplant in turkey said...

Thanks for the information. I really enjoyed, I would like get more
information about this,because is very beautiful, thanks for sharing
Fue Saç
Diş Doktoru

Op Dr Ali Mezdeği said...

Site's character and a great color match .. I will recommend your site to the other platforms.Saç Ekimi

Unknown said...

Now, the question is, who is up to the task of replicating the UN HDI's work on individual US states? Moreover, who is up to the task of editing the numerous incorrect Wikipedia entries?Jinekomasti