Getting off the Gold Standard

This is not a rant about either the strengths or limitations of RCTs. If that is what floats your boat, then you are in for a treat – there are a ton of posts and papers discussing the nitty gritty of the validity and efficacy of RCT evidence out there (see the “randomistas” Duflo, Banerjee, Angrist and Pishke, and Kremer, and for criticisms see Deaton, Cartwright, Pritchett, and Unlearning Economics)

My bugbear is merely with the phrase “Gold Standard”. What does it mean? When I hear “Gold Standard”, I envision something like this:

Lego race

Who assigned the Gold medal? In most of the literature, “Gold Standard” is put in quotations, yet there is never a reference to who has given out this standard. As far as I am aware there is not a ratings agency of evidence, at least in the sphere of social policy, giving out stickers to different research techniques? Or is there an FDA of social policy that decides on hard and fast rules on how to judge evidence?

I see the appeal of using “Gold Standard” as a classifier. It conveys credibility without overstating and calling RCTs the “best” research method (although, the association between gold and winning means that many people will substitute “Gold Standard” with “Best” while reading).  “Gold Standard” does erroneously imply consensus, and that there is an authority conveying standards. It also implies that other research methods are inferior and have been given silver or bronze standards. A conclusion is often drawn that RCTs are the “Gold Standard”, whatever the context.


But in trying to think of alternatives to “Gold Standard” –

  • “one of the best”
  • “a strong competitor”
  • “a mighty fine research technique”
  • “darn good”
  • “exceptional in some contexts and inappropriate in others”

–  you soon realise how difficult it is to succinctly describe and apply a standard to research methods.

Screen Shot 2017-09-11 at 10.20.02Also, “Gold Standard” always reminds me of a gold standard exchange system, something that is pretty much unanimously rejected now (except for some nuts Republicans who think a pretty metal is the answer to all our problems). This is not something that I want brought to mind when trying to think about rigorous and reliable evidence.

I will leave you with the advice of Rachel Glennerster, who wrote my favorite book on randomized evaluations and has written on the Generalizability Puzzle of evidence. She has also just announced she will be leaving J-PAL, where she advised that training of investigators dropped the use of “Gold Standard” as a descriptor of RCTs, to be the Chief Economist for DFID. Her approach is to remember that there are multiple strategies for gathering evidence and that it is always best to:

“Choose the tool that fits the question”

Screen Shot 2017-09-09 at 09.57.12

The lady doth protest not enough


I have not been to a demonstration since the anti-war protests in London in 2003. That demonstration was ultimately ignored and 14 years later I am certain this demonstration will be ignored as well.

The demonstration today outside Downing street was not on the same scale as the Iraq war marches but it lacked the clarity of those protests, which will make it easier to dismiss. Was this a demonstration about Trump? Theresa May? Immigration? Capitalism? All of the above?

I went to the march out of disgust at the Executive Order on Immigration that Trump signed and the lack of forceful condemnation from the UK government of such a prejudiced and counter productive policy . The only thing that I think a protest on the streets of London could possibly influence is the UK’s government response.

The young crowd outside Downing Street had more eclectic and wider reaching aims. There was a carnival atmosphere to the proceedings and I couldn’t help imagining what the civil rights movement marches were like, and comparing their stoicism with the jamboree of crude jokes on cardboard that was before me on this chilly January eve.

Chanting will never be my thing. The crowd mentality of giving up your choice of words to someone else seems to stop analytical thought.

“His hands are small, he can’t build a wall.”

“Build a fucking fence around Mike pence.”

“Say it loud, say it clear, refugees are welcome here.”

Chants that would fade in and out, peppered with lone shouts of “Fuck Trump” and “Dump Trump”. I find it bizarre that so many UK citizens are calling for a democratically elected leader of another country to be removed.

The anti-Iraq war movement failed. Have we learned anything from that process? How can we ensure that such a popular swell of public opinion is not ignored again?

The arbiters of success for demonstrations such as the one tonight cannot be attendance numbers and vague notions of solidarity. It has to be their ability to bring about change and the current method of marching has not been effective at changing anything – neither policy nor minds. Yet many people will go home from the march tonight thinking to themselves what a successful nights work.

If there is one thing we can learn from the civil rights movement it is the importance of thinking incrementally. So many protestors tonight came with broad, noble and naïve aims. We need more clarity and specificity. The focus at this juncture for British protestors should be for a stronger statement from Theresa May condemning the Executive Order on Immigration.

P.s The most annoying chant of the night was a call to free Melania and Ivanka Trump. I hate the hypocrisy of belittling and exonerating these women and making an assumption that they have no agency, from behind a faux feminist shield.

The Twenty Commandments


In these uncertain times….scrap that, all times are uncertain. In these turbulent times, when economics and the social sciences have taken a pounding due to their inability to adequately predict the future or designate risk, it is useful to remember Dani Rodrik’s Twenty Commandments from his book Economics Rules: The Rights and Wrongs of the Dismal Science:

Ten commandments for economists

  1. Economics is a collection of models; cherish their diversity.
  2. It’s a model, not the model.
  3. Make your model simple enough to isolate specific causes and how theyr work, but not so simple that it leaves out key interactions among causes.
  4. Unrealistic assumptions are OK; unrealistic critical assumptions are not OK.
  5. The world is (almost) always second best.
  6. To map a model to the real world you need explicit empirical diagnostics, which is more craft than science.
  7. Do not confuse agreement among economists for certainty about how the world works.
  8. It’s OK to say ‘I don’t know’ when asked about the economy or policy.
  9. Efficiency is not everything.
  10. Substituting your values for the public’s is an abuse of your expertise.

Ten commandments for non-economists

  1. Economics is a collection of models with no predetermined conclusions; reject any arguments otherwise.
  2. Do not criticise an economist’s model because of its assumptions; ask how the results would change if certain problematic assumptions were more realistic.
  3. Analysis requires simplicity; beware of incoherence that passes itself off as complexity.
  4.  Do not let maths scare you; economists use maths not because they’re smart, but because they’re not smart enough.
  5. When an economist makes a recommendation, ask what makes him/her sure the underlying model applies to the case at hand.
  6. When an economist uses the term ‘economic welfare’, ask what s/he means by it.
  7. Beware that an economist may speak differently in public than in the seminar room.
  8. Economists don’t (all) worship markets, but they know better how they work than you do.
  9. If you think all economists think alike, attend one of their seminars.
  10. If you think economists are especially rude to noneconomists, attend one of their seminars.
Rodrik is likely to become much more popular as demand for understanding the tensions that come from globalization rises.


The problem we all live with

Norman Rockwell, The problem we all live with, 1964

A truly memorable episode of This American Life from last year – The Problem We All Live With – that explores the beneficial power of desegregation in U.S schools. In particular the recordings from the town meeting that show white middle class families fears of black children coming into “their” schools is a harrowing listen.

I went to a private school as part of Thatcher’s Assisted Place scheme, which gave means tested vouchers for private schools to parents of children who scored high on entry tests. I’ve met a few people who benefited from the Assisted Place scheme and we are all middle class and white. Fewer than 10 per cent of the selected children had fathers who were manual workers, compared with 50 per cent in service-class occupations such as teaching, and that although children from single-parent families made up the largest category, other disadvantaged groups, notably the unemployed, and black and Asian families, had poor representation (Prof. Sally Power. Tracking the progress of Assisted Place holders thirty years on).

I’m grateful for the education that I received from a prestigious private school that my father, as a single parent, would not have been able to afford had it not been for the Assisted Place scheme. Yet I can’t help but feel that I was segregated away from the diversity of this country in a private school. I do not think I was able to benefit from and contribute to a school that represents the community that I grew up in.

Not that this will likely be a choice my income will give me, but I will never send any children I may have to an elite private school. I would send my child to private school if it was affordable for a family with a median family income. But I say this now, and I fear I could become one of those parents at the school meeting in the episode who have no regard for any child other than their own. Nikole Hanna-Jones discusses the moral quagmire of how to decide the right school for your child in this enlightening NYT article.

“Saying my child deserved access to “good” public schools felt like implying that children in “bad” schools deserved the schools they got, too. I understood that so much of school segregation is structural — a result of decades of housing discrimination, of political calculations and the machinations of policy makers, of simple inertia. But I also believed that it is the choices of individual parents that uphold the system, and I was determined not to do what I’d seen so many others do when their values about integration collided with the reality of where to send their own children to school.”

I will try my hardest not to uphold a system of segregation with the choices I make for any children I may have.

Albert Einstein


“If you have followed my meditations up to this point, you will probably wonder about one thing. I have spoken fully about what spirit, according to my opinion, youth should have instructed. But I have said nothing yet about the choice of subjects for instruction, nor about the method of teaching. Should language predominate or the technical education in science?

To this I answer: in my opinion all this is of secondary importance. If a young man has trained his muscles and physical endurance by gymnastics and walking, he will later be fitted for every physical work. This is also analogous to the training of the mind and of the mental and manual skill. Thus, the wit was not wrong who defined education in this way: “Education is that which remains, if one has forgotten everything he learned in school.”

For this reason I am not at all anxious to take sides in the struggle between the followers of the classical philologic-historical education and the education more devoted to natural science.”

– Albert Einstein, On Education, 1931