Getting off the Gold Standard

This is not a rant about either the strengths or limitations of RCTs. If that is what floats your boat, then you are in for a treat – there are a ton of posts and papers discussing the nitty gritty of the validity and efficacy of RCT evidence out there (see the “randomistas” Duflo, Banerjee, Angrist and Pishke, and Kremer, and for criticisms see Deaton, Cartwright, Pritchett, and Unlearning Economics)

My bugbear is merely with the phrase “Gold Standard”. What does it mean? When I hear “Gold Standard”, I envision something like this:

Lego race

Who assigned the Gold medal? In most of the literature, “Gold Standard” is put in quotations, yet there is never a reference to who has given out this standard. As far as I am aware there is not a ratings agency of evidence, at least in the sphere of social policy, giving out stickers to different research techniques? Or is there an FDA of social policy that decides on hard and fast rules on how to judge evidence?

I see the appeal of using “Gold Standard” as a classifier. It conveys credibility without overstating and calling RCTs the “best” research method (although, the association between gold and winning means that many people will substitute “Gold Standard” with “Best” while reading).  “Gold Standard” does erroneously imply consensus, and that there is an authority conveying standards. It also implies that other research methods are inferior and have been given silver or bronze standards. A conclusion is often drawn that RCTs are the “Gold Standard”, whatever the context.

2789996399_38ef2d33f4_b

But in trying to think of alternatives to “Gold Standard” –

  • “one of the best”
  • “a strong competitor”
  • “a mighty fine research technique”
  • “darn good”
  • “exceptional in some contexts and inappropriate in others”

–  you soon realise how difficult it is to succinctly describe and apply a standard to research methods.

Screen Shot 2017-09-11 at 10.20.02Also, “Gold Standard” always reminds me of a gold standard exchange system, something that is pretty much unanimously rejected now (except for some nuts Republicans who think a pretty metal is the answer to all our problems). This is not something that I want brought to mind when trying to think about rigorous and reliable evidence.

I will leave you with the advice of Rachel Glennerster, who wrote my favorite book on randomized evaluations and has written on the Generalizability Puzzle of evidence. She has also just announced she will be leaving J-PAL, where she advised that training of investigators dropped the use of “Gold Standard” as a descriptor of RCTs, to be the Chief Economist for DFID. Her approach is to remember that there are multiple strategies for gathering evidence and that it is always best to:

“Choose the tool that fits the question”

Screen Shot 2017-09-09 at 09.57.12

The Twenty Commandments

charlton-heston-as-moses-from-the-ten-commandments-1956

In these uncertain times….scrap that, all times are uncertain. In these turbulent times, when economics and the social sciences have taken a pounding due to their inability to adequately predict the future or designate risk, it is useful to remember Dani Rodrik’s Twenty Commandments from his book Economics Rules: The Rights and Wrongs of the Dismal Science:

Ten commandments for economists

  1. Economics is a collection of models; cherish their diversity.
  2. It’s a model, not the model.
  3. Make your model simple enough to isolate specific causes and how theyr work, but not so simple that it leaves out key interactions among causes.
  4. Unrealistic assumptions are OK; unrealistic critical assumptions are not OK.
  5. The world is (almost) always second best.
  6. To map a model to the real world you need explicit empirical diagnostics, which is more craft than science.
  7. Do not confuse agreement among economists for certainty about how the world works.
  8. It’s OK to say ‘I don’t know’ when asked about the economy or policy.
  9. Efficiency is not everything.
  10. Substituting your values for the public’s is an abuse of your expertise.

Ten commandments for non-economists

  1. Economics is a collection of models with no predetermined conclusions; reject any arguments otherwise.
  2. Do not criticise an economist’s model because of its assumptions; ask how the results would change if certain problematic assumptions were more realistic.
  3. Analysis requires simplicity; beware of incoherence that passes itself off as complexity.
  4.  Do not let maths scare you; economists use maths not because they’re smart, but because they’re not smart enough.
  5. When an economist makes a recommendation, ask what makes him/her sure the underlying model applies to the case at hand.
  6. When an economist uses the term ‘economic welfare’, ask what s/he means by it.
  7. Beware that an economist may speak differently in public than in the seminar room.
  8. Economists don’t (all) worship markets, but they know better how they work than you do.
  9. If you think all economists think alike, attend one of their seminars.
  10. If you think economists are especially rude to noneconomists, attend one of their seminars.
dani-rodrik13
Rodrik is likely to become much more popular as demand for understanding the tensions that come from globalization rises.

 

Two sides to education in Liberia

“For there is nothing either good or bad, but thinking makes it so.”

– Shakespeare, Hamlet

11-20the20necker20cube
The Necker cube is an optical illusion first published as a rhomboid in 1832 by Swiss crystallographer Louis Albert Necker.

Stare at the red dot and unless you are obstinately single minded your perspective will change. Stare long enough and you will be able to move between perspectives – you will be able to hold two views of how to see the cube and consciously switch between them. The yellow face is on the top of the cube. The yellow face is on the back of the cube. After deliberating I hope you have the faculties to see that both these statements are true.

I have recently been reading many opinions on the recent education reforms that are taking place in Liberia. This year, the government has started a project that aims to improve the failing education system with the introduction of a pilot scheme under which private companies and NGOs will take over the day-to-day running of 120 primary schools (3% of the total).  Similar to any scheme which mixes the private with the public, controversy has loomed large.

I was reminded of the Necker cube while reading two opinion pieces on the project. The first by Ben Phillips at ActionAid, describes a visit to one of the schools that is now run by one of the private companies taking part in the project.

screen-shot-2016-09-22-at-09-28-32

Bridge International Academies (BIA) has quickly become a darling of the Right and a Lucifer of the Left. Phillips goes on to describe a visit to the school and describes that class sizes have been kept smaller than they were before; that teachers are unprepared for lessons;  and that materials that have been promised have not been delivered to the school. All compelling evidence that private contractors in education cannot be trusted. All evidence that the yellow face is on the back of the cube.

If you contrast Phillips’ account with this article by Kevin Starr, the director of the Mulago foundation who has provided venture capital to BIA, you start to see another face to the cube.

screen-shot-2016-09-22-at-09-28-46

Starr sees evidence of cheap schools, with strict management, closely guided lessons, improved use of technology, and 8 hour teaching days. All evidence that the yellow face is at the top of the cube.

Empirically you can say the cube is across from you, just as you can empirically say it is below you. There is evidence for both views. Neither is wrong. But it is false to state it is only below you or only across from you. This stops being empiricism and becomes dogmatism. Similarly it is dogmatic to see private schools as either an innocent panacea or a malevolent virus for education systems.

Not enough people in this debate are striving to maintain the ability to see that attempts at privatization can be both good and bad simultaneously just as the necker cube is both across from you and below you. Not enough people are trying to weigh the costs and benefits of the approach simultaneously. It’s like both sides are loading their dice and imploring to you to play with their dice as they have weighted it the “right” way.

The poor quality of education in both the public and private sectors in Liberia is the real challenge, not the false dichotomy between the two.

 

Bill Gates

3019834-poster-p-1-the-most-important-idea-bill-gates-ever-had“I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal. … This may seem basic, but it is amazing how often it is not done and how hard it is to get right.”

– Bill Gates

Build a School in the Cloud

Mitra in this video argues that the information revolution has enabled a style of learning that wasn’t possible before.

We are only just starting to see the possibilities that a connected world will bring to international education.  Access to a world of infinite information has changed how we communicate, process information, and think. Decentralized systems have proven to be more productive and agile than rigid, top-down ones. Innovation, creativity, and independent thinking are increasingly crucial to the global economy. Where does this leave teachers?

A study at UC Berkeley demonstrated that kids given no instruction were much more likely to come up with novel solutions to a problem. “The science is brand-new, but it’s not as if people didn’t have this intuition before,” says coauthor Alison Gopnik, a professor of psychology at UC Berkeley. But who is going to give kids problems to solve? The teachers role is becoming less and less about crowd control and more about setting broad tasks. More about researching how their students learn and solve problems they don’t know the answers to. More problem poser than problem solver. This will make many teachers nervous and feel like their authority will be irrevocably eroded, but the potential for engaged learning and increased problem solving is huge.

“Never tell people how to do things. Tell them what to do, and they will surprise you with their ingenuity.”

– George Patton

Sensitization

[sen-si-tuh-zey-shuh n]

noun

  1. The state or process of being sensitized.
  2. The process of becoming susceptible to a given stimulus that previously had no effect or significance.

At least two times per school term the teachers from the elementary school I worked at in The Gambia would rally themselves for a “sensitization scheme”. A “sensitization scheme” consisted of the teachers walking from compound to compound in the neighboring villages and greeting parents accompanied by some vague enquiries about whether the children of the compound were going to school. I remember asking one of the teachers the purpose of the sensitization scheme and his tautological reply was that it was to sensitize the community. He also pointed out the added bonus that they might get a parent to donate a chicken for the teachers’ lunch bowl, but he couldn’t, nor could I, give a clear definition of the meaning of sensitize.

By using the word sensitization it privileged the word over the action. I am not questioning the use of complexity in language, but I often feel that buzzwords, such as sensitization, whilst having the intention to capture the complexity of how to bring about change, often distract from the purpose of action. I am doubtful that the buzzword of sensitization provided a boost and sustained the conviction of the teachers in their action that a “talk to the community” or “educate parents” scheme would not have been able to provide.

Cornwall and Brock (2005) have written on this topic in their paper What do Buzzwords do for Development Policy? A critical look at “Poverty Reduction”, “Participation” and “Empowerment” and reading it made me consider whether I have the faculties to critically analyze the topic of buzzwords as I live a life so couched in buzzwords that I may not be the able to see the wood for the trees. Every time someone asks what I am studying at Berkeley I have to give some explanation of development, which makes me consider whether development is itself a buzzword. Is the word development a method of exclusion or is this explanatory process a way of making development more inclusive? Does development have to become part of the common lexicon before it can be inclusive? Will we look back at the word ‘development’ in fifty years time and consider it bizarre that we had to explain its meaning to so many, or will we look back at it, as we look back at “civilizing” or “third world”, as a paternalistic relic of the past? We may have the chains of equivalence of development practice hanging around our necks but we must make sure we are not learning the new and old vocabularies of the model of development just to “reinforce status and widen the gap between expert and novice” and be wary that our use of buzzwords is not merely technocratic but transformative.

The Pitfalls of Evaluating Teachers

“Education is not preparation for life; it is life itself”

– John Dewey

Banerjee et al (2008) in their paper “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India” admit that the most expensive and important resource in a school is the teaching staff. But I find their contention that lack of teacher effort is the main problem afflicting schools in Uttar Pradesh disparaging and lacking empathy to the constraints faced by teachers.

The premise that teacher absence is based on a lack of incentives and motivation often slips into painting teachers as lazy individuals that are dragging education systems down. I also remember this caricature of the teacher constantly asleep at their desk only to wake up to beat their students with a ruler from “Poor Economics” by Duflo and Banerjee. I can only talk from my experience in a potentially very different context, and although I have come across high classroom absence rates amongst the teachers I worked with, those absences were rarely caused by laziness. Teacher absences in the school I worked at in The Gambia were for myriad reasons: unreliable travel infrastructure, inefficient government bureaucracy, commitments to attend birth, marriage and death ceremonies, Malaria, only being able to buy shoes from a weekly market, and runaway goats. Yes, lazy and unmotivated teachers exist, but absence from the classroom in many contexts is rarely a symptom of an unwillingness to teach.

One proposal has been to only hire teachers from the local community to teach in local schools but while this may combat some issues of motivation, monitoring and travel constraints, the unintended consequences of a system that prohibits the freedom of movement of a professional teacher would far outweigh the benefits.

Their contention that teaching is not an innately difficult task is predicated on the successful teaching of one skill by a trained volunteer over a short period of time. Proving that one small, albeit crucial, element of the job of a teacher can be done by a volunteer with minimal training does not prove that the job of a teacher is easy. Teaching in a low income country in the global south is not easy – large class size, a disparate spectrum of abilities and knowledge, inadequate infrastructure, inconsistent societal norms on corporal punishment, a wide ranging curriculum, and heat all contribute to the difficulty – and this difficulty is not adequately rewarded.

Should we argue for a systemic reform of what can be expected of teachers in rural schools? Ken Robinson has argued for a paradigm shift in “Western” education systems that were built on an industrial model and many education models in LIMIC’s are modeled on this same industrial paradigm of education. Teachers, as well as students, are victims of this paradigm. Shifting the paradigm is necessary, particularly in rural schools, as the constraints and needs are so different but it won’t be easy and it must be embraced at both the government and community level.