A mod points to a presentation explaining why the user in question was banned.

Removal Explanations

Reduce repeated rule-breaking

Our confidence rating

Likely

Share This Intervention

What It Is

Provide an explanation to users when their post or comment is removed for rule infraction.

Civic Signal Being Amplified

Welcome
:
Ensure user safety

When To Use It

When a user's post or comment breaks a rule that necessitates its removal, particularly a rule that is specific to the forum.

What Is Its Intended Impact

By providing users an explanation for why one of their comments was removed, you can reduce the likelihood that their future comments will break a forum's rules—and thus also require removal by moderators.

Evidence That It Works

Evidence That It Works

Jhaver et al. (2019) looked at the behavior of Reddit users who had at least one of their posts removed over a three month period and compared how likely they were to break a subreddit's rules again (i.e. have another post removal) depending on whether:

  • They received an explanation for why their posts were removed;
  • The explanations were brief "flare" message or longer comments; and
  • The explanation comments were delivered by a bot or moderator.

They found that users who received any explanation were less likely to have another post removed, and even less likely if the explanation was delivered as an extensive comment as opposed to a brief "flare". It made no difference, however, if a message was delivered by a bot or a moderator.

The limitation of Jhaver et al.'s study is that it is observational, so we cannot know if there is a confounding variable that may explain why explanation receivers are less likely to re-offend (e.g. the types of subreddits that provide explanations may tend to have users who respond positively to any removal). A second study, by Srinivasan et al. (2019), uses a "delayed feedback", "quasi experimental" design that gives us more confidence that removals with explanations positively affect users' behavior. They look at the subreddit ChangeMyView which provides comment explanations when a post is removed, but they take advantage of the fact that there is often a multi-hour delay before a post is removed. They find that users were less likely to transgress rules after having a post removed compared to "matched" control users who had not yet had their rule-breaking comment removed. The authors' intent was to study the effect of removing a comment, but they note that ChangeMyView also provides explanations as well as information about how to appeal the removal, and that they have a three strikes rule, all of which may be contributing factors to the positive impact of comment removals. The combination of Jhaver et al.'s observational finding about the effect of explanations and Srinivasan et al.'s natural experiment suggests to us that explanations are a promising intervention.

It is important to note that Srinivasan et al. did not see a decrease in toxic content; this suggests that removals and explanations are an effective way to make users aware of the specific rules of a platform rather than to disincentivize toxic behavior more broadly.

Why It Matters

Special Considerations

Regardless of whether they are given by bots or moderators, explanations might only work at reducing repeat rule-breaking if the forum in question:

  • has strict or unusual rules (i.e. not more common rules against toxic behavior), as they serve to teach new users about rules they may not be aware of;
  • also includes information about an appeal process; and
  • has a "three strikes" policy

Examples

This intervention entry currently lacks photographic evidence (screencaps, &c.)

Citations

Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit

Shagun Jhaver, A. Bruckman, Eric Gilbert
Proceedings of the ACM on Human-Computer Interaction
November 7, 2019
10.1145/3359252

Content Removal as a Moderation Strategy

Kumar Bhargav Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, Chenhao Tan
Proceedings of the ACM on Human-Computer Interaction
November 7, 2019
10.1145/3359265

Is this intervention missing something?

You can help us! Click here to contribute a citation, example, or correction.

Further reading

Back to Prosocial Design Library