Tag Archives: NGOs

University Rankings are Terrible. Now Can We Stop Doing the Same Thing With Countries?

The release of the US News & World Report college rankings is as good an excuse as any to talk about the sheer ridiculousness of organising complex institutions into rank order and pert decimal scores.

The criticisms of the index itself are nicely summarized in this Atlantic article, but for those that don’t have time for the full Gladwell, they are basically of two kinds:

  1. The ranking is flawed. The methodology constantly changes, schools juke their stats, it’s based on bullshit surveys that only measure the school’s established reputation, etc. The data might be good enough to distinguish Harvard University from  Southern Methodist Tech, but there is too much noise to say that Yale is better than Princeton or that Oregon State is better than Penn State.
  2. Most of the information we use to determine the quality of education isn’t readily measurable. How good are the teachers? Do students get enough personal attention? Is the campus social life welcoming or cliquey? If you list everything that made your college experience positive or negative, you won’t find it in the number tables of these rankings.

A few years ago I was working at a human rights NGO, and one of my jobs was preparing reports for multinational corporations telling them about the human rights situation in countries where they were thinking of operating.  You want to open a shoe factory in Kenya? Here’s what you need to know about gender discrimination, corruption, occupational health and safety. Here’s how you make shoes there without violating human rights.

Sometimes companies would ask us for big packages of countries, 10, 20, 50 at a time.

‘Can’t you just give us a ranking?’ they would ask. ‘Tell us which country is the worst of that list, and we won’t make shoes there.’

Or, they would suggest, better yet, give us an index. If you tell us that Bolivia scores 8.2 out of 10 and  Iran scores 8.3 out of 10, we can make our decision on quantitative data rather than just putting our finger in the air.

‘But that is putting your finger in the air,’ I would tell them. ‘It’s just us doing it instead of you. Bolivia and Iran, their politics, their demographics, their economics, they look nothing like each other. Two numbers isn’t going to make that go away.’

This is one of my beefs with the Failed States Index, the Economic Freedom Index, the Human Development Index, the Corruptions Perceptions Index, the dozens of other indices that purport to rank countries according to some difficult-to-measure variable.

The problem is, countries have all the same problems as colleges. The data out there isn’t strong enough to justify precise determinations, only broad tranches. Yeah, Norway is less corrupt than Angola, but I don’t need an index to tell me that.

But is Norway less corrupt than Denmark? More developed than Switzerland? Given the limited data and even more limited number of indicators these indices use, the answers to those questions are a re-statement of your methodology, not a useful analysis of the conditions in those two countries.

I used to try to tell the companies this, that any attempt to rank countries according to their ability to prevent corporate human rights violations would be like trying to rank kittens according to cuteness. After you separate them into the already-obvious tranches, it’s just a judgement call, preferences disguised as data.   

‘But it would be so much easier for me if you could do that anyway.’ Only one corporate person ever actually said it this directly, but afterward I started hearing it, in subtler ways, from the others.

Eventually I realized that the only reason the companies pushed so hard, why they insisted so strongly on  rankings and scores over information and analysis, was because it made it not their problem anymore. They didn’t have the credentials to pull 50 ‘good’ countries from 100 uncategorised ones, so they used us to push the responsibility away. ‘It’s not me saying Bolivia is an 8.2,’ they could tell their boss. ‘A human rights NGO said it was. Making shoes there is totally approved. 

I don’t know if high schoolers use college rankings to decide where they should get educated. And I don’t know if multinational corporations use country indices to decide where they should make shoes. I just hope that in both cases, they know that most of what they’re seeing is either totally obvious or entirely unsubstantiated.

26 Comments

Filed under America, Personal, Work

Hand Over Fist

So I’m eight months into my first NGO fundraising job. I’m enjoying it, but the translation of genuine human need into effective marketing sometimes makes me feel cynical and complicit.

We were having a discussion the other day how to best communicate our issue in few words and strong images. We’re trying to strike a balance between enticing people to donate and ensuring that we aren’t manipulating them or blowing our issue out of proportion.

Convincing people to support your organization isn’t the same as selling them a bicycle or a spatula. There are actual human beings at the receiving end of the work we do, and I think that gives us an obligation for truth, sobriety and maturity in our communications that we don’t share with conventional marketers.

And then there’s Unicef:

You can imagine some bespectacled 30-something at a consulting firm (in fucking Brooklyn, undoubtedly) going, ‘You! Get me a picture of the cutest, saddest African baby alive! … And you! What’s the most tragic five words you can imagine? I want ’em in all caps!’

I’m not offended or disappointed by this, exactly. Unicef’s a great organization, and if we all spent 200 bucks a year supporting them instead of updating our iPods or whatever, the world would probably be a better place. It’s just funny, in an inevitable sort of way, how marketing turns everything it touches into camp.

Or in other words, don’t hate the player, hate the game. Unicef is officially a ‘competitor’, so I clearly need to rise to this standard. Is that baby available for a photo shoot in Berlin? I’ll contact his agent.

1 Comment

Filed under Random, Work