ADI IGNATIUS: I’m Adi Ignatius.
ALISON BEARD: I’m Alison Beard, and this is the HBR IdeaCast.
ADI IGNATIUS: Alison, it’s been interesting to live through the evolution in economics. In the 70s and the 80s, you had this early explosion of behavioral economics led by people like Daniel Kahneman, Richard Thaler. And then 20 years ago was the Freakonomics phenomenon. So you had Steven Levitt, an economist, and Stephen Dubner, a journalist, who wrote a book that popularized all of this thinking that attempted to show the hidden side of everything, what truly motivates us as economic actors. And the field took off behind this basic tagline that conventional wisdom is wrong. Are you a fan?
ALISON BEARD: Yeah, absolutely. I mean, that was the book that made economics cool. Everyone wanted to read it, everyone wanted to replicate it. And I do think it changed the way that companies think about consumer behavior and also the way that managers thought about getting the best out of their employees.
ADI IGNATIUS: Yeah, I think that’s right. I think it opened our eyes to a lot of phenomena that were not immediately clear. I think it also changed academia in some ways, that professors saw the success of Malcolm Gladwell’s books, of Freakonomics and thought, “Well, wait a minute, I can do research that is also broadly relevant, that has popular appeal, and I can make a name for myself.” So I think the book has actually had a profound impact.
So 20 years later, I spoke to Stephen Dubner, the journalist part of the duo, and we talked about the book’s legacy. We talked about what they got wrong, what they got right, and why and how we can find the hidden side of everything. So here’s that conversation.
ADI IGNATIUS: All right, so Stephen, welcome to the HBR IdeaCast.
STEPHEN DUBNER: Thank you so much. Appreciate it.
ADI IGNATIUS: So congratulations, the book is 20 years old. It has had an outsized influence, I’d say, on American culture, on global culture. Levitt, I guess, was an early member of that generation of economists that kind of shook up the profession, making use of data sets that either didn’t exist. We were sort of into the data era at this point, or were inaccessible or didn’t seem seemly for economists to use that really looked at what truly provoked consumer behavior, individual behavior. So how did the economics profession respond to him and then to him and you when the book came out, and especially when the book sold so many copies?
STEPHEN DUBNER: I think there was a range. So there were some people who were irate with Levitt. There were people who felt that he was showing how the magic, the regression analyses get performed. And I think they weren’t so happy about that.
And then there were a lot of economists that we started to hear from maybe six to 12 months later who recognized that the success of Freakonomics was actually really good for them too. And it came in a couple forms. One was my mother, or my mother-in-law maybe more often, has no idea what I do. Thank you for writing a book that she can read that sort of explains it. So that was one.
The other was economists recognized that since this was a book that became very popular and it did have a foundation in how an academic economist goes about doing his work, that it would probably drive more students to economics at universities. And whether our book did that or it happened to coincide with a big surge in economics majors, I don’t know, but there was a big surge and therefore it did, I would say, enlarge and maybe enrich the market for economics professors.
So I think anytime you do anything that’s high profile, it doesn’t even have to be popular, but high profile, a lot of people, their first instinct, they’re almost trained. If you’re a journalist, if you’re an academic, if you’re a certain kind of cultural critic, you’re almost trained to look for the flaws or to look for the ways in which it can’t be as good as everybody’s saying it is. So I get that.
Why it captured the public’s attention the way it did is hard for me to explain. I have a lot of half-baked theories, probably none of them are very correct, but I mostly dwell in the gratitude space and then let’s get on with it. How can I use this opportunity, this platform, to do more work that I like to do? And Levitt felt the same for him.
ADI IGNATIUS: So part of the appeal of the book, and really the point of the book is that there’s a hidden side to everything. But isn’t that also risky and almost conspiratorial? I mean, aren’t the best explanations usually or often simple and straightforward?
STEPHEN DUBNER: I think a lot of times they are for sure. We tried to point out when the conventional wisdom is “wrong” or maybe partially wrong, we tried not so much to just play Gotcha and say, “Oh, look at you being a simpleton for thinking X causes Z like that.” What we tried to do is look at the whole ecosystem around that particular piece of conventional wisdom and look at how it was created and who were the experts who created and published and promulgated that conventional wisdom.
Did they have their thumb on the scale or a horse in the race in any way? The answer to that question is almost always, yes. I feel we’ve always been pretty positive about the systems, even when we critique them, because humans are, I think we’re a pretty great species, but we’re fallible. So even when we mean well, we often mess up a little bit. We might have an intention to create some great piece of policy, some pro-social policy, maybe it’s in the education space or healthcare space. And then when you gather the data 10 years later, you see, “Oh wow, not only did it not produce the gains we thought, but it actually kind of backfired. There were some unintended consequences.” But you don’t go throwing those charges around willy-nilly. You wait until there’s data and you try to measure the effect.
ADI IGNATIUS: Without any evidence, I would say that the success of Freakonomics, the success of Malcolm Gladwell’s writing, and I interviewed Malcolm on this podcast a little while ago, was partly a cause of kind of a reenergizing of the economics profession, and certainly the behavioral economics profession. But I also think there’s a crisis in that area in social science research, and certainly in business and workplace related research where you’re seeing a slew of retractions fabricated and shoddy data, and then this whole network of right of self-appointed arbiters who are ready to try to replicate and to denounce the scholarship.
So I think you may have unwittingly contributed to that because you made this stuff accessible, because people realize if you produce an interesting counterintuitive finding, you’re going to get press coverage, you’re going to get attention, you’re going to sell books. And that’s created an incentive that if you’re a skeptic or a cynic, you’re thinking, “Well, people are putting their finger on the scale with the data to try to come up with these things.” I’d be interested in your take on that because unintentionally you’re part of this kind of new world of scholarship that can be more popular and can be more lucrative. So talk about that and the pressure to get it right given those circumstances.
STEPHEN DUBNER: I mean, first of all, I would say that the incentives to get attention for yourself or your work are as old as humankind. If you look at yellow journalism from whatever, 100, 130 years ago, and I came up as a journalist. So all these axioms are drilled into your head dog bites, man, ho-hum. Man bites dog, great. Local TV news, if it bleeds, it leads. Is it representative at all? No, it’s not representative at all. When I came up as a print journalist, I was at the New York Times. We like to say that TV is called a medium because it is neither rare nor well done. So all of us have been-
ADI IGNATIUS: Well, and then remember when USA Today came out, somebody said this is for people who find TV news too complex.
STEPHEN DUBNER: So the incentives to get attention like that, whether you’re a producer or a producer of news, a producer of research, et cetera, I mean, they’ve always been very, very strong. It’s funny because people did single out little tidbits from our book, and they did become cocktail party fodder. Did you know that blank, blank, blank. And it’s funny, we didn’t treat it like that when we wrote it. I think the first chapter began with this story about a fine gone wrong at an Israeli daycare center where parents were coming late. So the school instituted a fine so that the parents would not come late, but the fine was relatively small. What happened was that more parents began to be late because they figured I could just pay this small fine, a few dollars and actually get in an extra game of tennis or whatever.
It’s like cheap childcare. That was the kind of story that people would love to retell us, which was fine. It was fine. But we told that story, which was based on a research paper by, I want to say it was Uri Gneezy and Aldo Rustichini. The whole point of telling that story was to give a really tiny example, a tiny scale example of a few dollars of how incentives work. So when people talk about incentives, especially economists, the first consideration is usually financial incentives. And we were trying to create a framework or an argument that there are all kinds of incentives that we’re all always responding to. I have a strong incentive to burnish or protect my reputation. There may be moral or social incentives that matter a lot to me, more than financial incentives. So I guess I feel like we weren’t just dropping one-liners to get attention, but this book of ours was full of stories connected to, like I said, bigger ideas or bigger themes.
In terms of the larger issue that you raise about academic fraud and the replication crisis and so on, the incentives of the researchers and the incentives of the universities to protect research that may be bad are so strong, and that’s because the universities are run by lawyers, and lawyers are extremely, extremely risk averse. So I hate, hate, hate the fraud. I will say this. We also covered the kind of whistleblowers and the do-gooders, if you want to call it, Ivan Oransky at Retraction Watch does amazing work. The three guys at Data Colada, they’ve done a lot and very selfless work and work that draws attention to them for being the whistleblowers, which is not what they want to do. There are a lot of people fighting the good fight there, but again, the incentives are massive.
There are hundreds, maybe thousands of essentially fake academic journals that exist to offer paid publication to scholars or would be scholars, but especially second or third tier scholars. So that’s a joke. It’s a terrible situation. Those same journals will also charge university libraries a huge amount to force them to carry the journal. So it’s very, very murky in there.
One thing that always attracted me to economists, I will say, is that I don’t want to say that economic research is prima facie more robust than psychology research or sociology research or anthropology research. But I will say this, the economists that I’ve interviewed and gotten to know over the years, which is probably in the at least many dozens, probably hundreds, and a lot of psychologists as well, at least in the many dozens, and I really respect a lot of their work.
The research feels very different. There’s a lot more data in an economics paper. If you really want to fake a massive data set, you could, but you’d be crazy to try because the way that the peer review works. The other thing I love about economics papers that I wish the other social sciences would copy is when you’re an economist and you write a paper for publication, you say, “Here’s my thesis, here’s my data, here’s my methodology, and here’s my chief finding. Here’s what I’m going to argue in this paper, and here’s how I’m going to argue it.” And then they will typically say, “Now I’m arguing the X causes Y, but there are others who might’ve thought in the past or might think in the future that A causes Y or B or C. I’m going to go through those and explain why I don’t think those are reliable answers or why they maybe carry a small piece of explanatory power, but not as large as a piece that I’m about to give you.” I find that to be an unbelievably useful exercise. It also takes two, three, four years to write a paper like that. So it’s a very labor-intensive thing.
ADI IGNATIUS: There is the basic question, causation versus correlation. And that seems to be where amateurs trip up or journalists trip up or journalists can get duped or something like that. And if you purport to show that the consumption of Pop Tarts is driving up the price of oil, I’ll run that story, but you probably have a causation correlation problem there. How have you learned to apply rigor to that question?
STEPHEN DUBNER: Yeah, it’s a great question, and it’s really hard. I mean, the more complex something gets, the harder it is to say with any kind of certainty that you know that X causes Y. And we do try to illustrate in the book, and certainly in radio programs ever since then, how easy it is to fall into a bad correlation, causation trap. One example we wrote about maybe in our second book was about polio, which was a really terrible but interesting disease, the way it was approached. And I believe the cause is still essentially unknown or the origin cause the way it spread came to be known. But once the vaccines were available, that question became less pertinent.
But there was a theory at a certain point since polio tended to spike in the summer, and a lot of people would keep their kids out of swimming pools and things like that. There was a theory that the consumption of ice cream caused polio similar to your pop tarts and oil maybe.
So you can see why, especially in the moment, especially if things are going sideways, especially if people are scared, you can see why people attach themselves to what seemed to be causal relationships that are only correlational at best. What we tried to do is really just show the homework.
So in probably the kind of most high profile and highest stakes example of a causational argument in Freakonomics was based on a paper that Steve Levitt written with John Donohue earlier about how the legalization of abortion led to a generation later a decrease in crime because it made available abortion to women whose children would’ve been born into circumstances that are more likely to lead them to trouble, lower income, unstable families, unstable circumstances. Interestingly, one thing that I think we wrote that people almost never attached to that or remember from that, is that it didn’t necessarily mean that that woman that would be mother wouldn’t have a child.
It was often a timing thing, and that a child would often come later when she was better set up to have a kid and to raise that kid productively. So it was really an argument about how unwanted-ness is a driver of an outcome or a life that is more likely to have trouble in it, whether it’s crime or whatnot.
But when we wrote that piece of the book, and this was really drawing on a very, very, very intense and robust analytical paper that Steve Levitt had written already with John Donohue, we again entertain all kinds of other potential explanations. And then you marshal what Levitt would call a collage of evidence to make your claim feel as if it’s about as substantiated as it can be. So for instance, one simple maneuver in that thought process would be to say, “Okay, if the legalization of abortion represented federally nationally by Roe v. Wade in 1973 led to a decrease in crime 15, 16, 18 years later, what do you do to get something outside that sample set of 1973 a switch is turned on?”
You’re always looking for a place where a switch might’ve been turned on earlier or later, or a different kind of switch was turned on, or something else came in to provide for you what they call an instrumental variable that’s clean. In the case of abortion, one piece of evidence from that collage of evidence was that I believe was five states had previously separately as states legalized abortion. So okay, now you’ve got a different instrumental variable to play with, and you can look at in a place like New York or California, did crime begin to fall earlier than it did elsewhere because there was availability to abortion earlier? And the answer was yes.
So when you’re trying to make an argument, whether you’re a politician or a business person, whatever, I like people who show their homework. And then when I’m interviewing them, I like to say, this is a very basic question. I would encourage everybody to ask all the time, what is your best evidence that the argument you’ve just made is true? And if you don’t have any evidence, then I can probably assume that you’re just BSing me or making it up. If you offer me evidence that is based on a survey on the street of 18 people, I’m going to say, “Well, that doesn’t sound very reliable either.” If you’re going to say your evidence is based on administrative data from a big state in India from the 1960s, I’m going to say, “Well, yeah, that’s a lot of data, but it’s from a long time ago.” So yeah, you got to poke and investigate the data as best as you can if you want to make an argument that something is actually causal. And that said, making causal arguments is really hard, and that’s why science is hard, and that’s why I like scientists.
ADI IGNATIUS: So one difference between now and when the book came out, when the book came out, you could disagree with the methodology. You could talk about correlation versus causality, but I feel like we sort of agreed then that there were verifiable facts that could be analyzed and picked apart. I don’t think we’re in a post fact era because I think we have more data at our fingertips than ever, but somehow in the political divide that we have, people are comfortable with the idea of there being alternate facts and dismissing inconvenient facts as somehow fake or somehow made up. I worry about that a lot as a journalist, as you can imagine.
But I guess I’m curious – you’re ultimately looking at some fact-based set of numbers, circumstances that can be analyzed, but is the world open to fact-based explanation at this point?
STEPHEN DUBNER: Yeah, I’m with you. It saddens me. It frustrates me. It scares me how easy it is for people to think whatever, to put it really basically that something is true if it’s demonstrably not true. Now, granted, it’s hard to prove that sometimes that something is demonstrably true. I guess I’ve responded to that by, look, I don’t think I’m the person that’s going to be able to fix that problem. I think there are millions of people in the world who are struggling to fix that problem. And I don’t know if they’re doing very well either, but it’s just not something that I think I have a particular talent for.
What I try to do is just practice what I kind of would like to be preached a little bit more, which is find the people who know the stuff, ask them questions that are fairly well-informed. I try to do a good bit of prep before I interview anybody about anything. I try to give them the room to really explain themselves.
I listen really hard so that if they say something that either isn’t clear or people misspeak all the time in interviews, and sometimes I don’t catch it. And then we’re looking at the transcript later and someone said when inside the curve, instead of when outside the curve and we’re like, “Oh, well, I guess we don’t want to use that tape.” But then we’ll call them up and say, did you misspeak here? And if they did, then we may rerecord them saying that. But that’s just basic journalism, basic fact checking. So that’s the world where I came up in which I still operate in, which I still like. But then I think it’s just the way you do your work and people either subscribe to that style or not. Our listenership of Freak Radio, it’s a very interesting listenership. It’s broad and it’s very diverse, I would argue. I mean, this is a probably unprovable argument. I would argue it’s one of the most diverse, I would call it a large niche media audiences in the world.
We’ve got about two and a half million people that listen to at least one of our episodes every month. And the emails that we get are from people, such a diversity of people doing so many different things, and the way that they think about the world is so different. And I take a lot of comfort in that because I think there is a large community out there of people who simply want to understand the world better, often with the intention of making the world a little bit better than when they came into it. And that’s me. That’s what I like to do too. So I’ll give you, for instance, we’re working on a two-part series right now about the air traffic control system in the United States. I’m sure that if I went on Reddit for two hours, I could come up with a story that would frustrate, scare, whatever, do a lot of things, and it would be based on probably very little fact, at least very little primary source fact.
ADI IGNATIUS: But a lot of passion.
STEPHEN DUBNER: A lot of passion, a lot of passion, and look, passion is wonderful. So what we did for that, and I have to say the producer on this, Theo Jacobs did a really, really, really good job of working through a lot of potential sources. And the sources that he came up with for me to interview are all extraordinarily substantial, smart and good at describing the scenarios they’re dealing with. So one of them is the CEO of a major airline. One of them is an economist who specializes in transportation, deregulation, and really knows the FAA inside and out.
These are people who, when you hear them being interviewed, I would argue that almost anybody that’s hearing or seeing this would say, “Okay, I get it. This person knows what they’re talking about. They’re not BSing, they’re not pushing a position, et cetera.” So that is a style of journalism or storytelling that I like doing. It’s fun. It’s interesting. I believe it when I publish it. I sleep well at night. And look, there’s a lot more room for that kind of journalism and storytelling in the world. So for anybody out there who feels like they’re locked into this thing where they have to come up with tiny little cocktail party things that are going to get them a new contract or an invitation to something, I would just say the water over here is good.
It takes a little bit more work. You have to adjust your expectations a little bit. I grew up on a farm as a kid. We used to make maple syrup and do all this outdoor work that was backbreaking and very labor-intensive. And I sometimes think of this kind of journalism as making maple syrup. You have to go around to all the trees, you got to bang in the taps, you put a bucket, you have to empty out the buckets when they fill up. Then you got to boil down all that sap, and you get this much at the end of the day. And I’m like, “That was a lot of work to get this much.” And that’s kind the way I feel every Friday morning when we publish an episode of Freakonomics Radio. It’s one little episode in a sea of things, but I feel it’s maybe not as delicious as maple syrup, but I think it tastes pretty good.
ADI IGNATIUS: So let me ask before we have to close. As you reread it, is there something, or are there things that make you cringe where you just think, “Wow, I would not have done if we were doing a book now? I wouldn’t do it same way.”
STEPHEN DUBNER: I think we wrote it now. The tone would be different. We had a really good time writing it. We were often playful in a way that I think now would feel a bit callow. We were also younger. I’d like to think that I’ve gotten a little bit wiser, kinder as I’ve gotten older. I’ve tried at least.
I would say there were very few errors that needed correction. One that did, and we corrected it early. It was painful because it meant taking down a hero. We wrote about this guy named Stetson Kennedy, who had done a lot of work against the Klan and against racism in the south at great personal risk to himself. He infiltrated these Klan meetings and Klan groups and so on. As it turns out, after we published the book, I heard from someone who had worked in some archives that intersected with this work on the Klan.
And it turned out that according to our best reading of all these archives that came after the publication of the book, that this guy was right. And so it seems as though he had exaggerated his activities or conflated his activities with someone else who was an undercover agent. And so that was really painful because I had to… he was an older guy. He since died, Stetson Kennedy, and he did a lot of very, very, very good work, but I couldn’t let that stand if he had exaggerated to that extent. So I called him up, I said I needed to talk to him about something serious. He said, “Come on down.” He was in Florida. I flew down there. We went to this place for lunch. And I just laid out the argument and the look on his face, I’ll never forget, it was a very unpleasant reaction to have been the cause of, but he also didn’t admit, he just kind of said, “I don’t know what you’re talking about. Everything I wrote is true.”
He’d written this probably 40, 50 years ago. But the more I tried to unpeel the evidence and present it to him and say, “This doesn’t match with this. This doesn’t match with this.” I was persuaded I was right. So what we did in that case was we wrote a column for the New York Times about how we had gotten this wrong, why we’d gotten it wrong, and what the truth was. It didn’t change the nature of the story, but it certainly changed some of the facts. And then we rewrote that section of the book and republished it for later editions. So look, nobody wants that to happen. Anybody who’s written anything that gets read by more than 20 people, you’ve probably had one of those 20 come and say, “That’s not quite right.”
So you do as much as you can to get it right. With every episode of Freak Radio, we research, we interview, we fact check, we fact check again, we still make mistakes. And then if we make one that gets into the publication, we republish. But I think in terms of Freakonomics, the book, I like it. I liked it a lot. I learned a lot.
I like that it’s out in the world and it makes people think. I love – you know, we’ve gotten thousands of emails or letters from people who have been, whatever, inspired either by our books or the radio shows to do amazing things. Kicked off a big kidney donation community, kicked off a lot of education and healthcare things. So I’m very, very proud and happy about that. And for anybody that became more cynical because of what we wrote, I’m sad about that, but we’re not cynics. Skeptics, yes, cynics, no.
ADI IGNATIUS: All right. Well congratulations on the 20th anniversary edition on keeping the franchise going for so long. And thank you for being on HBR IdeaCast.
STEPHEN DUBNER: It was a lot of fun. I liked your questions. Thank you very much.
ADI IGNATIUS: That was Stephen Dubner, host of the Freakonomics podcast and author of Freakonomics: A Rogue Economist Explores the Hidden Side of Everything.
Next week, Alison talks to executive coach Muriel Wilkins about something else hidden: the blockers that hold leaders back. If you found this episode helpful, share it with a colleague and be sure to subscribe and rate IdeaCast in Apple Podcasts, Spotify, or wherever you listen. If you want to help leaders move the world forward, please consider subscribing to Harvard Business Review. You’ll get access to the HBR mobile app, the weekly exclusive Insider newsletter, and unlimited access to HBR online. Just head to HBR.org/subscribe.
Thanks to our team: Senior producer Mary Dooe. Audio product manager Ian Fox. and Senior Production Specialist Rob Eckhardt. And thanks to you for listening to the HBR IdeaCast. We’ll be back with a new episode on Tuesday. I’m Adi Ignatius.
