It was a mixed bag.

I think steelmanning succeeded in the sense that it seemed to me that the person that felt the most attacked, initially, felt more empathized with after the steelman. This was one of the original goals of steelmanning:

We hope that steelmanning improves relations between people and helps achieve the best possible world for everyone.

However, this success seemed hollow in the end because the empathy didn’t seem to be reciprocated, and I think this is the main reason the debate failed. I spent over one hundred hours creating the steelmen for and against childhood spanking but the person responded with comparatively minimal effort, and ultimately ended the debate prematurely.

I don’t regret the time I invested. I learned a lot about the evidence for and against different parenting philosophies, statistical nuances in correlational research, and the limits of scientific research on certain questions.

However, I was disappointed that steelmanning didn’t accomplish more.

Despite the failure, I also discovered a key improvement in steelmanning which is the concept of the burden of proof that should be in every steelman.

The idea of the burden of proof is to try to describe what one side would need to see in order to re-consider their point of view. Although the burden of proof may have controversial, non-scientific premises (e.g. ethics, etc.), it hopefully outlines how an argument can be resolved, even if that’s ending the argument with a clear understanding of the other person (Walton, 1988):

One of the most trenchant and fundamental criticisms of reasoned dialogue as a method of arriving at a conclusion is that argument on a controversial issue can go on and on, back and forth, without a decisive conclusion ever being determined by the argument. The only defence against this criticism lies in the use of the concept of the burden of proof within reasoned dialogue. Once a burden of proof is set externally, then it can be determined, after a finite number of moves in the dialogue, whether the burden has been met or not. Only by this device can we forestall an argument from going on indefinitely, and thereby arrive at a definite conclusion for or against the thesis at issue.

It may be no coincidence that the other person ended the debate when I was exploring their burden of proof.

Other lessons learned:

1. Ultimately, if another person has little motivation to debate something seriously, it seems the style or content of the argument doesn’t really matter. In that sense, steelmanning was a success because it helped clarify the other person’s level of motivation.
2. After presenting an initial steelman, if the other side doesn’t immediately explore the specifics of it nor reciprocate the steelmanning, then it’s easy to fall back into unproductive debates. Ideally, one would steelman every new topic that’s brought up, but this is incredibly difficult. One solution might be to observe that the debate is veering into other topics and propose that either:
   1. If they think the other topic is critical, both sides can go off and do steelmen of that other topic. It’s important that both sides do it so that both sides feel empathized with.
   2. Or, if they think the other topic is not critical, go back to the original steelman, starting with a focus on the burden of proof.
3. It’s possible that a fully empathetic style of argumentation such as steelmanning may plant a seed. The other person hopefully won’t have a distasteful memory of the argument and this positive memory might help them reconsider things in the future.

Finally, another personal lesson is that I will be careful in what I decide to steelman in the future because it is such a large investment of time. That always includes my other priorities at that time, but it may now include the importance of the topic to me, the closeness of my friendship with the other person, and gauging how motivated the other person is.

As far as the topic of childhood spanking itself, some might be surprised to find that it seems that the scientific evidence is ambiguous. As summarized in the meta-comments section of the steelmen by Bauman & Friedman (1998), my review of the evidence in the 25 years since their paper suggests the same conclusion (of the scientific evidence):

The inevitable conclusion from a critical, objective review of the scientific research on corporal punishment is that the data are inadequate to permit a conclusion on either its effectiveness or its negative consequences.

The controversy started after “a mother who reviews this data on my personal time” reported an error about the reported death rates on Twitter to the study’s main author, Dr. Flaxman (Kelley, 2022; Kelley, 2022b). Dr. Flaxman admitted to the error on Twitter (Flaxman, 2022). The study was subsequently fixed and re-published, and significantly changed the results: “We have fixed an error” (Flaxman et al., 2022c).

One might think, “that’s unfortunate but that’s just science in action”. The paper was used by the CDC prominently, but one might think, “that’s just an unfortunate mistake in a fluid situation and the vaccine would have been approved anyway due to other evidence”.

However, some facts about the controversy raise some questions:

1. It was a paper about the United States that was authored by a team mostly out of the United Kingdom. The team made basic mistakes mixing data from incompatible databases which is something a U.S. team accustomed to such data would not do, particularly the CDC itself which publishes the raw data.
2. The paper originally mixed cumulative COVID-19 deaths and annualized deaths for other causes. No study was used to justify this (Flaxman et al., 2022d), and it was removed in the final version. However, the CDC knowingly mixed annualized and cumulative deaths as it noted on its slides (U.S. CDC, 2022b; U.S. CDC, 2022c).
3. The paper was a pre-print meaning it was not yet peer-reviewed. This is not an inherent problem but suggests extra caution.
4. The claims were repeated at least three times by three different CDC doctors: Dr. Daley, Dr. Fleming-Dutra, and Dr. Oliver (U.S. CDC, 2022f; U.S. CDC, 2022g; U.S. CDC, 2022).

Here’s the steelman defending the CDC:

1. The CDC are under extreme pressure in an emergency situation with limited resources so they depended on an external team of scientists and didn’t have the time to investigate it in detail due to competing demands for time. The lead author, Dr. Flaxman, has co-authored other research on COVID-19 epidemiology in the United States so he was being trusted (Monod et al., 2021).
2. Despite the cumulative statistic ultimately being removed from the study, cumulative mortality is a plausible way to compare COVID-19 deaths to other deaths because lockdowns, masks, and social distancing might have reduced the death rate that otherwise would have occurred. The use of the cumulative statistics were clearly noted on the slides.
3. Even if COVID-19 ranks lower than flu & pneumonia, there are still a significant number of COVID-19 deaths for this age group and the basic point stands that childhood deaths are a significant justification for approving the vaccine as the flu vaccine is similarly approved for this age range (U.S. CDC, 2022e).

Here’s the skeptic steelman:

1. Instead of analyzing their own data which is not particularly complicated (U.S. CDC, 2022d), the CDC used the analysis of a non-peer reviewed study by a team from the United Kingdom who were so unfamiliar with the data that they made a basic and significant mistake that, when fixed, largely changed the story the CDC presented at the vaccine approval meeting. The fixed study shows half the rates as seen in the CDC slides, ranked COVID-19 7th instead of 4th/5th, and ranked flu & pneumonia as worse mortality than COVID-19 (Flaxman et al., 2022b).
2. The CDC knowingly used an uncommon comparison of death rates (cumulative vs. annualized), later removed from the study, that had purely speculative assumptions which were not even mentioned. The original study reported annualized COVID-19 statistics but the CDC consciously chose to use the cumulative statistics. The presentations also emphasized that COVID-19 was the top infectious cause of death which is no longer true in the study results.
3. The CDC could have presented mortality much more simply, gotten the point across that COVID-19 kills lots of children, and the vaccine would have still likely been approved, so what was the point of such gratuitous exaggerations? At best, this is a sign of general incompetence; at worst, motivated reasoning and vaccine zealotry. In either case, this makes one wonder about the CDC’s other data and interpretations.

We reached out with questions to Dr. Flaxman, Dr. Daley, Dr. Fleming-Dutra, and Dr. Oliver but have not heard back for days. If they respond, we’ll update this post.

Below are the slides the CDC used during its presentation to the approval committees (U.S. CDC, 2022; U.S. CDC, 2022b; U.S. CDC, 2022c).

Mainstream: For children aged 6 months through 4 years, COVID may cause serious medical issues including death, the Pfizer vaccine is likely effective in reducing COVID acquisition, its estimated benefits, including reducing community transmission, outweigh the estimated risks of side effects across the population, vaccination may have benefits in addition to natural immunity, vaccination in older groups is associated with reduced deaths, and the burden of proof is on those potentially jeopardizing their children and their community

Skeptical: For children aged 6 months through 4 years that are healthy, risks of COVID-19 may be similar to the flu (influenza virus), the Pfizer vaccine is not likely very effective in reducing COVID acquisition and will likely wane quickly, it is in a new class of vaccines with limited safety data including more severe adverse reactions in the vaccine group compared to placebo, one child death in the equivalent Moderna vaccine trial in children under 6 compared to none in the placebo group, double the serious adverse event rate in the vaccine group compared to placebo in the equivalent Moderna vaccine trial and 24% higher all-cause mortality in the equivalent Pfizer adult clinical trial, community benefits are unclear, population-wide risk/benefit calculations do not account for individualized assessments and value judgments, natural immunity is likely robust, and the burden of proof for net benefits is on vaccine providers

For details of the arguments, review the full steelmen.

Steelmanning

As a refresher of our first post defining steelmanning, it is another name for the principle of charity in argumentation that tries to make the strongest possible argument for someone by empathizing with their positive intentions. Steelmanning gets its name as the opposite of strawmanning. Strawmanning is an argumentation fallacy in which a person’s argument is made into a man of straw – easy to knock down – and then this strawman is argued against instead of what the person really meant.

Tinmanning

Before we begin, let’s define another term: Tinmanning.

Tinmanning is a fallacy when someone declares that they’re steelmanning but they’re actually strawmanning.

Summarizing the terms

• Strawmanning: Making a reasonable argument unreasonable.
• Steelmanning: Making an argument stronger.
• Tinmanning: Someone saying they’re steelmanning when they’re actually strawmanning.

Steelmanning may be condescending, harmful, or arrogant

Ozy Brennan writes that when someone declares that they’re steelmanning, they’re usually tinmanning and the declaration of steelmanning may be condescending, harmful, or arrogant:

In the least obnoxious case […] not only is this person strawmanning you, but they’re also acting like you’re an idiot and they’re so much better than you for being able to think of the argument you actually made.

[…]

In the most obnoxious case, Alice doesn’t actually understand Bob’s argument at all. Often, there are fundamental worldview differences. […] Instead of understanding that people believe things differently from you, you’re transforming everyone into stupider versions of yourself that don’t notice the implications of their own beliefs.

[…]

You can say “but neither of those are actually steelmanning! Real steelmanning is being able to put other people’s viewpoints in words they themselves find more compelling than their own arguments!” However, that is an extraordinarily rare and difficult skill; even most people who do it once can’t do it consistently. Saying “to steelman position X…” should be interpreted the same way as saying “to express perfect loving kindness for all beings…” It’s certainly a nice ideal which people might want to approach, and some people even manage to pull it off sometimes, but it’s a bit arrogant to declare that you’re definitely doing it. Even when you think you are, you usually aren’t.

Steelmanning Brennan: Even if a person is well-intentioned, steelmanning is very hard to do and thus often leads to tinmanning. Tinmanning may be condescending, harmful, or arrogant. Therefore, declaring that one is steelmanning usually fails and causes unnecessary damage. A better approach is to switch from the debating approach of steelmanning to a collaborative truth-seeking approach, understanding actual viewpoints, and seeking out well-informed advocates.

Response: All forms of truth-seeking are hard and may lead to strawmanning which may lead to being condescending, harmful, or arrogant. The benefit of steelmanning is that it explicitly tries to avoid strawmanning through a foundation of empathy. By analogy, science is very hard, but it’s better than the alternatives. The proposed alternative (truth-seeking collaboration, understand actual viewpoints, seek out well-informed advocates, and finding common ground) seems to be part of proper steelmanning.

For more details of this argument, see the Steelman Anything topic.

Steelmanning one argument may strawman another

Dr. Gelman argues that steelmanning one argument may strawman another:

[Steelmanning] can lead to being uncharitable or “strawmanning” of other positions that are being opposed or caricatured by the people you are steelmanning.

Dr. Gelman’s proposed alternative is:

to try to address the arguments as [they] arise.

Steelmanning Dr. Gelman: If person A is steelmanning an argument of person B, person A might accept one of person B’s premises as part of steelmanning (i.e. empathizing “for the sake of argument”). If person A didn’t otherwise believe this premise and another person C considers this premise a strawman, then steelmanning introduced a strawman that Person A wouldn’t have otherwise done if they were just “addressing the argument as it was”.

Response: Accepting any strawman while steelmanning – even of someone who’s not necessarily involved in the argument – is a failure of steelmanning; instead, it’s tinmanning. Person A should have also steelmanned Premise P1. In contrast, “addressing an argument as it is” is not necessarily designed to reduce strawmen. Even if such an approach did not include Premise P1, it may have strawmanned other premises.

In response to the above argument: Steelmanning an argument so that all people do not consider that it contains any strawmen may be impossible due to peoples’ contradictory premises. Even if we grant in the above example that, theoretically, Person A should have steelmanned Premise P1, there may be no way to steelman it in such a way that both Person B and Person C are happy.

Response: While it may be impossible to steelman some arguments in a way that everyone accepts, if the goal is to empathize with other people for the purpose of improving relations between all people and helping achieve the best possible world for everyone, then there is no better method than steelmanning (see the all truth-seeking is hard response), and we should try our best to steelman as much as possible.

For more details of this argument, see the Steelman Anything topic.

Ironmanning: Making an unreasonable argument reasonable

For completeness, ironmanning is when someone makes another person’s “unreasonable” argument reasonable. This is the converse of strawmanning. What makes something “unreasonable” on its face is a subjective judgment, so this is not necessarily a fallacy like strawmanning or tinmanning.

In our opinion, accusations of ironmanning aren’t very useful unless the accuser is willing to debate what is “unreasonable” and why they judge the subject so.

Steelmanning gets its name as the opposite of strawmanning. Strawmanning is an argumentation fallacy in which a person’s argument is made into a man of straw – easy to knock down – and then this strawman is argued against instead of what the person really meant. Straw conveys weakness because it’s made from dry plant stalks whereas steel conveys strength because it’s a strong metal alloy.

The main reason we like steelmanning here is its attempt to empathize with other people instead of just trying to win arguments. We hope that steelmanning improves relations between people and helps achieve the best possible world for everyone.

What is Steelman Anything?

Steelman Anything is the movement to try to make the strongest arguments for all sides. Its main home is at https://steelmananything.com/ which is an organized reference and the above definition of steelmanning comes from the introduction topic.

Subscribe to this blog using its RSS feed.

All of the content of Steelman Anything is open source on GitHub and we welcome issue reports and contributions. All content is licensed in the public domain with the CC0 license.

Who is the main author?

My pen name is Lucius Asclepius.