Seeking Ferret

lirazel recced the podcast "Sold a Story" by education journalist Emily Hanford, about the way reading is taught in American schools and why she believes a particular pedagogical strategy for teaching reading, called 'three cuing", is wrong and is harming students.

Some semi-organized observations on listening to the podcast are below. I went back and forth on whether to post this, it may be nonsense. I want to be very clear that this is not an essay on whether Hanford is 'right' about the science or not. I don't know and I don't really have an interest in doing the work to find out- I'd have to do months or years of literature review and possibly independent research, some of which is not realistic to actually carry out. This is an essay critiquing the rhetoric of Hanford's podcast, the rhetoric about how science works and how we should think about competing scientific arguments. This is an essay not about science but about the stories we tell about science.


1) There is a story we tell about science that is almost always wrong. The story goes that we used to do things a certain way, then SCIENCE told them that they were wrong, and then we changed to follow science or left the anti-science people to their backwards ways. I've just built up an automatic hackles generator around this story. This story can be told about a scientific idea I agree with, it can be told about a scientific idea that I don't agree with, and either way, I object to the story. This story underestimates Popper and Kuhn and all the philosophy of science thinkers we look to when we try to understand what science is and how it works. This story is never the story they're telling, this is not what 'paradigm shift' means. This story tends to be wrong in a number of ways, but perhaps most importantly to me, this story almost inevitably underestimates the people who were 'wrong'.

If you dig into any story written in this form in any detail, you will find that the people who were wrong were not entirely wrong, and the people who were right were not entirely right. This is an ironclad rule in my observation (though of course there probably are exceptions. As I said, the story is 'almost always wrong' My epistemic uncertainty about this is high.). And what is worthwhile and interesting to dig into whenever you come across a story in this form is why did the people who believed the thing that is wrong believe it? What evidence were they using and what misled them? To what extent were the people who were wrong right, and what part of their ideas should we try not to discard when we discard the stuff that is wrong?

But post-Enlightenment people like this story of science overthrowing ignorance, and they keep telling it, over and over again. And they particularly like it when it agrees with their intuitions about the world, and they can use it to argue that the people who disagree with them are 'anti-science'.

Emily Hanford's podcast is all about this story, and so I object to her podcast on those grounds. It doesn't matter whether she's right or not about three-cuing, she's wrong to tell this story about science.

(Here are a couple examples from my dreamwidth of me attacking this story from different angles in the past Against Newtonian Mechanics and Against Deutschephysik and Against Aristotle.)


The version of the story that Hanford tells is something like this:

Starting in the mid 1970s, neuroscientists started using new technological tools and new experiment designs to study children learning to read. They did a variety of experiments. Using MRI technology, they tracked the brain pathways used when children were reading. They tracked the brain pathways used when the children were using different reading strategies. They did observational studies of children reading. And by the late 80s they'd built a body of research that said clearly that the way in which people read quickly and fluently is based in large part on sounding out words mentally. Based on this research, they concluded that children learn to read by learning how to sound out words, so therefore the best way to teach students to read is to teach them the principles of English phonics.

However, at the same time as this research, many teachers were being trained to offer their students a different strategy called three-cuing, a strategy that prioritized figuring out unfamiliar words from context as a stepping stone to figuring out how to read. This strategy was 'wrong' and resulted in some students who otherwise would have learned to read, instead struggling to read. However, teachers were told that they simply needed to have faith that these students would eventually learn to read this way and because the teachers were not aware of the neuroscientific research they believed their trainers.

There are immediate problems with this story. I mean, I'm going to get into specific problems later, but there are immediate narrative problems. This isn't how science works in the social sciences. We don't just magically find that there is One True Answer to a human problem that a simple controlled experiment or even group of experiments proved incontrovertibly. Human brains don't work like that. This is at the heart of the so-called 'replication crisis', where in some fields half of all published results cannot be replicated by a different group trying to do the same experiment.

Purely speaking on intuition, a better version of this story just inherently goes, schematically, "Following the science, teachers debated multiple competing theories of how to teach children how to read. Each strategy had virtues and flaws, none of them were 100% successful at teaching, and research continued on how to best balance these competing virtues." But let's think a little more specifically.


1A) Systematic Errors I

I think we need to be particularly careful here about systematic errors- errors in the way we interpret all of the data. There are repeated anecdata throughout Hanford's story about individual students who were not learning to read by the three-cuing method, but who then learned to read once they got alternative tuition in phonics. Little Zoe was not able to read, her parents eventually realized this and got her phonics training and then she was able to read.

That all may be well and good, but let me present a counter-anecdote that I think illuminates something important. When I was in third or fourth grade math, I was ineffectively taught the long division algorithm and I never understood it. I was quick thinking and had developed reasonably good mathematical intuitions, so I was able to fake it to a certain point by kind of just running factors in my head until I got the right one, and similar tricks. But at a certain point the numbers I was being asked to divide got too large to do in my head. I was in honors math in sixth grade and I didn't know how to do long division and at some point it came to a head and I was struggling. So I went to my father and tearfully confessed to not being able to do long division, and he spent ten minutes teaching me how the algorithm worked and I never had a problem again.

It could be that my father, who is a smart man but not particularly gifted mathematically (I surpassed him by about ninth grade), explained the algorithm better than my third grade teacher had. I don't really think that's the case. I think they explained more or less the same thing in more or less the same way. I think the difference is that I was in sixth grade instead of third grade, and I was developmentally further ahead, so I was now capable of understanding what had eluded the younger me. And every one of Hanford's anecdotes could have the same explanation: First Grader Zoe couldn't learn to read by any method; Second Grader Zoe could learn to read no matter what the method.

That doesn't mean that's what happened. It really could be that Hanford is right and the reason young Zoe learned to read is because she was finally taught phonics. But it ought to mean that you shouldn't make the argument on the basis of Zoe. You make the argument on the basis of studies that control for age and look at many students. But controlling for these things is hard! No matter how much you do it, you can never be completely certain that there wasn't some factor in your experiment design that led to it showing a result that won't scale when you try to deploy it across the country. And so instead people resort to anecdotes.

1B) Systematic Errors II

There's a fundamental problem with using the modern technology of the scientific method to achieve universal goals like 'all students should be able to read'.

Not all human brains work the same way.

I'm giving that its own paragraph. There's a lot of talk about the 'replication crisis' in the social sciences, and there are several different factors that all flow into the replication crisis including misapplied math, but this is probably the most important. If you drop a ball off a building in a gravitational field, it will always fall the same way. If you drop a human off a building in a gravitational field, they will react differently every time. Humans are not predictable. Humans do not always behave the same way. And anyone who tells you that humans are predictable 'in aggregate' is ignoring away the outliers.

If you hypothesize a pedagogical strategy for teaching reading that has been scientifically validated to successfully teach 90% of students to read in six months, consistently, that still means vast numbers of children not learning to read with that method. But reason would still say that that 90% strategy is miraculously amazing, and ought to be 100% pedagogically recommended. 90% is way better than we're doing now with any particular strategy. It's reasonable shorthand to say "Science says that children should be taught reading by learning phonics" if the well-designed studies all say that it's 90% successful. But you can't forget about the other 10%! In semiconductor manufacturing, if you get 90% yield on a production step you throw a party and toss the other 10% in the bin. In education, you can't toss children in a bin. You need to remember to be doing something else with the other 10%, or 20%, or 50%.

I did not leave Hanford's story convinced that phonics is a 100% solution. I definitely didn't leave it thinking that three-cuing is a 100% solution. I left it thinking that, assuming that all the facts Hanford reports are correct, that three-cuing is a 60% solution and maybe phonics is a 75% solution. So this is another way of thinking about the story of little Zoe. Perhaps there's nothing wrong with three cuing, plenty of students learn how to read that way, but Zoe is simply one of the children it doesn't work for. Again, I don't know that. It could be that Zoe could learn to read if only she were taught phonics, but it also could be that Zoe could learn to read if only she were taught phonics, but that doesn't mean that the rest of the class, who are already successfully learning to read, should be taught phonics. I think this is a fundamental error of scientific analysis, to say that because some students failed to learn using a particular teaching strategy the method should be entirely discarded. Unfortunately, that will never be right. I say unfortunately because it would be great if we did have a single implementable strategy that worked for every single student.


2) There is a story about the history of universal education that is worth thinking about. I'm not an expert on the history, but give or take a few decades, every child being in a classroom and being taught to read only dates to the 1920s, and white people caring that every child in the classroom actually successfully learns to read only dates to the 1970s and the civil rights movement. So we're only talking a history of about fifty years of really trying to teach everyone to read. (As a side note that I'm not going to address in much detail either, race is an implicit subtext in so much of Hanford's story, but it's never an explicit text and I think this is really egregious. The reason this debate between phonics and three cuing exists, it should be made clear, is because in the 1970s the educational establishment decided that it was important to teach black students how to read. As soon as we decided that every child meaning every child should learn to read, suddenly statistics could exist showing that we were failing to meet that goal.) That's incredibly recent on the timescale of these things.

That's not a thing Hanford mentions in the podcast, but it seems incredibly important to me. In most versions of The Scientific Story I mentioned earlier, there is some earlier Wrong Science to be overthrown. The geocentric model, Aristotelian gravity, whatever. What's absolutely wacky about this story is that there's no, like, 19th century theory that the entire establishment subscribed to. Hanford doesn't really talk about what came before. As I understand it, it was some combination of phonics and just exposure to reading that was the dominant strategy. Most children who learned to read more or less just learned to read. Either that or they didn't learn to read and were routed to a life that didn't require them to read.

A fundamental idea behind a controlled experiment in anything involving an intervention in humans is to demonstrate an improvement relative to the status quo. If a drug is better than not taking a drug, but it's not as good as a different drug, there's no reason to switch to the new drug even if it works. So defining the status quo carefully and correctly is essential to demonstrating a result is an improvement. And Hanford doesn't do this. or quote any scientists who do! There's no "Back in the 1960s before Lucy Calkins, everyone learned to read just fine." There's a good reason for this. I don't think she can do this. I don't think there's any idyllic past point in history when all children learned to read more easily than they do now. So it feels overly optimistic, not to mention misleading, to argue that if only we taught phonics, every child would learn to read. There's no historical basis for making that assumption. There are, fact, numerous moments in the faddish educational history of the past 60 years where phonics was taught more than whole language pedagogy, and still there were plenty of children who didn't learn to read.

So this story is inherently different than the Scientific Story in this regard. When Hanford tries to argue how egregious it is that teachers are not taking advantage of The Science, it's not like she's claiming that they're clinging to an old, non-scientifically derived theory. She's claiming that they are also doing science, they're just not being effective. They have studies, too! High powered, effectively conducted studies! She just thinks they don't say what their proponents claim they do.

Kuhn claims there's two kinds of scientific investigation, 'Normal Science' and 'Paradigm Shift'. In Normal Science, scientists all agree on the paradigm and they use evidence to argue back and forth about which theory better explains the evidence. In Paradigm Shift, the new science uses such radically different vocabulary and conceptualization that arguing and evidence can't be part of the process. Hanford seems to be arguing that phonics is a paradigm shift driven by a new kind of neuroscience research, when it seems to me that the question of phonics vs. whole word pedagogy is just a debate in Normal Science.


3) There is a story about the pandemic and how it changed education. Or maybe I should say ruined education. I've talked to lots of teachers about their life in the pandemic and none of them felt good about the experience. Education simply fell apart for a couple years. Teachers weren't prepared to effectively teach over zoom. They had trouble monitoring student behavior, trouble keeping students actively in the classroom, trouble evaluating their work.

I'm seeing it professionally now- we're bringing in college age interns and post-grads who are a year or two behind where pre-pandemic interns were educationally. They simply had a couple years of not really learning anything.

In some cases, students just fell behind entirely. In other cases, parents shifted their lives to fill some of the gap and try to support students. So part of Hanford's story is about parents, suddenly exposed more closely to the pedagogical choices guiding their childrens' education, having stronger opinions about it than before. I don't know that there's anything wrong about that, it's probably good for parents to be aware of what their children are being taught and how, but parents are not experts in pedagogy and there's plenty of reason to suspect that for nonintuitive questions of pedagogical strategy, teachers have reasons for what they do that are deeper than parents can see instantly.

And I think this raises important questions that the story doesn't go into, like, if we're going to involve parents in education the way we did during the pandemic, shouldn't they be given some training? Or at least explanation and transparency on how teachers think about teaching. Obviously asking them to go to a multiyear teaching program to become parents is probably untenable or at least dystopian, but maybe there is more that can be done so this doesn't come as a surprise to parents. But also parents are not the masters of their childrens' education, that's dumb, that's the engineer's disease and I know that because I'm an engineer and this whole post is the engineer's disease. :P I do not go around telling farmers how to farm, I do not go around telling bankers how to bank, I do not go around telling teachers how to teach.

But also I think that there's a deeper narrative problem with looking at the failures of the pandemic years and deciding that any kind of educational strategy is failing. The bottom line is that very few kids actually learned anything during the pandemic years. Motivation was impossible, accountability was impossible, learning was impossible. And not just because of the lack of in person school. Because people were dying around them and the adults were scared, because adults were rearranging their lives and therefore rearranging the lives of their children, because it was impossible to take school as seriously when real life was happening around them. We all know this, we all watched it happen, we know the kids are not okay. So it is utterly crazymaking to me to hear people say " I thought my kids were learning to read during the pandemic, but it turns out they weren't, because the pedagogy was wrong." The pedagogy was not the problem.

Which is not to say that Hanford's argument rests entirely on the failures of pandemic education, but just that the mix of narratives of pre-pandemic and pandemic eras doesn't tell the right story for me.


3) There is a story about parenting.

There's an incomprehensible amount of pressure on parents to be the masters of their childrens' education, to know everything and be everything and guide everything, because education isn't just in the classroom, it's how parents read to their children at night, how they answer the endless stream of curious questions their children will throw at them, how and how often their children interact with other children. Because only the parents are in the position to control that, there is pressure for them to control all of that, and it's just deeply unrealistic. It's especially unrealistic if we're going to throw a 'science of reading' frame on top of everything, because now parents need to read and evaluate hundreds of neuroscience papers, half of which are total bullshit, and figure out what is meaningful. If parents already aren't trained as teachers, how can we expect them to be trained as neuroscientists? And even the neuroscientists in Hanford's story acknowledge times when they allowed their children to fall behind in their education.

If Hanford's story has any value, I hope one of the ways is in just recognizing that part of the answer is to just accept that the educational process will, at some point or another, fail every child. And that's okay. Children are resilient, children have variable needs, and variable interests and missing one opportunity does not mean missing all opportunities. You just figure out what isn't working and try to find something else, and parents do not need to self-flagellate when they fail, they just need to keep trying.


4) There is a story about teaching effectively that is different than a story about pedagogy.

There's a subtext I kept wishing Hanford would do more with, and eventually in the final episode she does a little more, but not enough, about how there's pedagogical theory and then there's lesson planning.

The Calkins method that Hanford criticizes sounds like it's actually mostly independent of the theory of three-cuing. It doesn't care how students learn to read, what it's actually trying to do is give teachers tools, at the level of lesson planning, to inspire students to enjoy reading. Why does it teach three-cuing? Because Calkins reached out to other experts at reading pedagogy and adopted the learning theory of three-cuing wholesale. And if three-cuing is 'wrong', then that's a problem. But what Calkins cared about, and is devoting her life to, is figuring out how to structure lessons in a way that get students excited about them. And it's no wonder that teachers are rabidly enthusiastic about her work. Hanford spends a lot of time in a rhetorical pose of bafflement that teachers are using a method that 'science' says doesn't work, but here is the explanation to resolve her bafflement: Teachers can readily see that the Calkins method works! Like, right in front of their eyes, students get excited about reading, in a way they don't when they teach phonics, at least the way they were taught to teach phonics. And don't you dare try to tell me that science says that students shouldn't be excited if you want them to learn. That doesn't pass the smell test.

Instead, Hanford tries to argue that if students get through the phonics, no matter how miserable it might be, they will become excited on their own because books will be unlocked to them and they can pursue their own joy. That's not a crazy story, but it's not self-evident. Again, this goes back to the novelty of universal education. In the 19th century, the self-selected privileged children who would be exposed to education understood that it would include drudgery. And if the students couldn't handle the drudgery, they'd drop out at age 10 or 12 and go into the workplace, or if rich enough, into idleness. But if you're going to try to teach everyone, everyone has to include the students for whom drudgery is miserable. Or you need mechanisms of compulsion. But thankfully American education has purged most of the worst mechanisms of compulsion. No more beating the children who don't read well.

So school needs to inspire children to want to learn. All the children. And the Calkins method, it sounds like, does that, and it inspires teachers to want to teach, and when they take that inspiration into the classroom they see that more of their students are doing the work. So that's the answer to Hanford's bafflement, if you try a teaching strategy and you can visibly see students learning, then obviously it has value.

And so I find it mildly infuriating that Hanford interviews a bunch of teachers who go around talking about their feeling of betrayal upon being told by Hanford that the Calkins method 'doesn't work.' It's such transparent gaslighting. Even if phonics is scientifically better than three-cuing, there's more to teaching than pedagogical strategy and more to a classroom than teaching. And the Calkins method clearly has all of those things going for it, so telling teachers that it doesn't work is tricking them into throwing out all the good.

Calkins, by the way, has abandoned three cuing in the last couple years. Her position is that she has learned better strategies for teaching reading than three-cuing, and so she is revamping her system to include the superior strategies. Of course she is! Because she doesn't care that much about pedagogical theory, what she cares about is giving teachers tools to inspire students. It was infuriating at times to hear this be self-evident in Hanford's interviews and not to hear her acknowledge it.

One of the things I deeply believe about teaching without any substantiation is that some small fraction of teachers are brilliant teachers who inspire students and who know how to provide them with useful presentations of information that help them learn and learn how to learn, who just have an incredible ability to see students where they are and give them what they need to grow. And most teachers are just normal human beings who do what they were trained to do and just try to make it through the work day. Giving those teachers consistent, straightforward, repeatable tools to inspire students is essential. Giving them 'the science of reading' might be less important than that.

There are a lot of small problems with Hanford's reporting, but to me this is the criminally dangerous part, the way her reporting works again and again to diminish the importance of inspiring students to want to learn and giving them a learning environment they are comfortable in and that their teachers are comfortable in. And not only to diminish the importance of that, but to make it sound like the tools of inspiration are cultlike and sinister.


5) There is a story about The Science of Reading that cannot be boiled down into Phonics is Good and Whole Word Pedagogy is Bad, and Hanford acknowledges in the last five minutes of her second bonus episode that this is true. Vocabulary training, contextual reading, reading comprehension, there's a whole lot of other skills involved in reading beyond word decoding and the Science of Reading has evidence about all of it, none of which Hanford had the time to cover in her podcast. And Hanford's two bonus episodes are full of sobbing parents excited to learn from the podcast that all they need is this One Weird Trick so that their non-reading child will be able to learn to read, and it's full of half-cocked legislators seizing on the podcast to meddle in local control of reading education, and Hanford knows this is bullshit and she says that it is bullshit, but it doesn't stop her from gloating over those sobbing parents and those half-cocked legislators.

And again, that doesn't mean that Hanford's overall point that more children will learn to read if they are taught decoding by properly trained teachers is wrong. It sounds mostly right to me, a guy who knows nothing about teaching reading. But so much about the way she tells the story makes me skeptical that it is the whole story. There is more to always more to science than the stories we tell about science.

When I heard the learn’d astronomer,
When the proofs, the figures, were ranged in columns before me,
When I was shown the charts and diagrams, to add, divide, and measure them,
When I sitting heard the astronomer where he lectured with much applause in the lecture-room,
How soon unaccountable I became tired and sick,
Till rising and gliding out I wander’d off by myself,
In the mystical moist night-air, and from time to time,
Look’d up in perfect silence at the stars.

~Walt Whitman

I first encountered this poem in high school English, and I come across it again every few years. I can't explain entirely the rage it summons in me.

But maybe this is the point I wish to make. A friend mentioned the Randall-Sundrum model of the universe and I went to that wikipedia page to try to learn what that was. Pretty soon I was desperately linkhopping- I have a basic education in relativity and differential geometry, but pretty basic, and even the vocabulary I did learn at some point, it's been a decade since and I needed to refresh my memory.

So I clicked on anti-de-Sitter space and from there to Lorentzian manifold and from there to Riemannian manifold, and I want to point out something about these four articles.

The article on Randall-Sumdrum model begins "In physics" The article on Anti-de-Sitter Space begins "In mathematics and physics." The articles on Lorentzian Manifold and Riemannian Manifold begin "In differential geometry." There's that tricksy slippage between physics and mathematics Whitman is writing about. Are the learn'd astronomer's "proofs, the figures," his "charts and diagrams" a meaningful and interesting representation of the actual stars, or are they just lifeless mathematical models that lack the "mystical" potency of observing the stars with the naked untrained eye? Aside from answering this question, though, the distinction is, I think, actually important to doing physics. Because if you theorize that spacetime takes a certain shape that can be modeled by a particular manifold, and then your measurements in an experiment don't match the manifold, you have to consider two different possibilities: One, that spacetime doesn't match your theorized model, and two, that your measurements were inaccurate. But if you're a mathematician working with a manifold and it doesn't match your expectations, only your math is wrong.

So this distinction Whitman writes on matters. There are the mathematical models of the stars, and there are the actual stars themselves, and if you forget this you end up confusing the manifold with the spacetime. A physicist needs both to do their work.

Nonetheless, I feel a great rage when I read Whitman's poem, a rage at the idea that the untrained eye bestows a more exciting and therefore truer reality than the subtle delver into the measureable mysteries of the cosmos can attain through experimentation and analysis. This may be dogmatic scientism on my part, but if so, let it be!

When we talk about Einstein overthrowing Newton, there's a degree to which the language we use is wrong. After all, as a mechanical engineering student, I had six or seven semesters of Newtonian mechanics and one semester of Einsteinian mechanics. Newton must have been really, really wrong, for his theories to still be taught in that depth a century after he was relegated to the dustbin.

If you drop an apple to the ground, it falls according to Einstein's equations, not Newton's (to the best of our understanding at the moment). But if you try to measure the position of the apple as a function of time, that measurement will look as if it follows Newton's equation just as much as it follows Einstein's equations. That's because the difference between Einstein's result and Newton's result is infinitesimal in the speed regime of the apple's fall, at such a small fraction of the speed of light.

So to a certain extent the choice of using Einstein's equations or Newton's equations in this regime is arbitrary. We could, every time we wanted to calculate the speed of that apple, use general relativity. The result might be closer to the physical reality. Then again, we know that Einstein is 'wrong' too by the same definition that says that Newton is wrong. We believe there are regimes in which Einstein's equations don't accurately model reality, and that there are other equations we don't yet have that do a better job of modeling the movement of bodies in these regimes, the so-called equations of the Grand Unified Theory.

And yet in spite of that, in spite of us knowing for sure that both sets of equations are inaccurate models of the universe, either Einstein's or Newton's equations can be used to calculate the motion of the ball with accuracy sufficient for any practical purpose. What's more, we can simplify those equations yet further- leave out forces like air resistance that both theories know are part of the math, for example- and still get useful results much of the time.


Saying that science is never settled, that scientists are constantly disproving past theories, misses this point. The value of science is not its elegance. The value of good science is its descriptive power, which endures even after good theories fall.

http://www.wnyc.org/story/reading-screens-messing-your-brain-so-train-it-be-bi-literate/

In which I am skeptical about research in the social sciences and the reporting thereof, news at 11.

Did you know that Twitter is rotting your brain? Well, I'm not on Twitter, so my brain's not rotting, but you, all of you who use Twitter, your brains are rotting away because of Twitter. Well, and also because... get this! : You have no gene for reading!!!!!!

"In the old days before the internet, reading was a linear event," Mike Rosenwald.

Except for all the times in which it wasn't. Like reading a newspaper with columns that you could jump between. Or flipping past a boring chapter in a book to get to an interesting one, or flipping to the end of the book to find out who killed Roger Ackroyd. Or walking down a busy main street looking at all the signs on storefronts. Or, you know, reading the Talmud.




"The human brain is almost adapting too well to the particular attributes or characteristics of internet reading," says Maryanne Wolf of Tufts University.

This is because unlike such skills as seeing, there is no gene in the human genome for reading, the story tells us. It is something scary called a learned skill, which means that the human brain is forced to rewire ancient brain circuits in order to read. But if you exercise those brain circuits in the WRONG WAY, meaning using Twitter, you will reprogram your brain so that you are not capable of 'reading linearly'. (In other news, sometimes it is hilarious when you take a metaphor and extend it too far just to see what happens.)

How do we know this twitter brain rot is happening? Because RESEARCHERS gave 25 people a story on paper, and 25 people the same story on a Kindle, and the ones who read it on paper were better able to describe the order of events in the story. "Significantly" better, in fact. Of course, somehow science reporters still have not figured out that statisticians use the word significantly to mean something entirely different than what the general public thinks it means. For a statistician, it means that there was a difference between the two groups that was not explainable by pure chance in the sampling. For the general public, it means that there was a difference between the two groups that was big enough that we should care about it.

In this case, the researcher has not yet published her result, only discussed it with reporters and presented it at a conference, so I cannot tell which it is, but I rather suspect it's the former. BECAUSE IN SCIENCE JOURNALISM, IT'S ALWAYS STATISTICALLY SIGNIFICANT, NOT ACTUALLY SIGNIFICANT.

To be fair, there's other proof. This one guy who works at the Washington Post thought he was reading less deeply than he used to, so he called a few friends and they said "Me too!". And then they all said "It must be the Internet!" I AM NOT JOKING, THIS IS PART OF THE NEWS REPORT.


I think it is likely that we read books differently than we do internet things, and I think it is likely that the things we choose to read are also teaching us how to read, but these intuitions do not extend to claims about how the internet has reprogrammed out brains so that we can't read deeply anymore. Because the brain is really complicated and we mostly don't understand it. And because I have read the first 11 books of the Wheel of Time on my ereader over the last two months, linearly.