Removing Rule-breaking Comments

Reduce rule-breaking and recidivism

Our confidence rating


Share This Intervention

Back To Top

What It Is

Deleting a comment that breaks a forum's rules while notifying the rule-infractor that their comment has been deleted.

When To Use It

If you are a platform or online community and you want to not only remove harmful content but also reduce rule-breaking.

What Is Its Intended Impact

To reduce the likelihood a rule-breaker will be a repeat offender and to reduce rule breaking among other users.

Evidence That It Works

Evidence That It Works

Ribeiro et al (2022) used a quasi-experiment design to determine the effect of deleting comments that broke Facebook's rules regarding the use of violent language. At the time of the study, Facebook used an algorithm to detect how likely a comment included violent language and automatically removed comments that passed a given threshold; comments below that threshold were not deleted but instead "hidden" in a thread, so that other users needed to click in order to see them. Users whose comments were removed were notified their comment broke a rule and was deleted, were told of the consequence (either a warning or 24 hour suspension) and given an opportunity to disagree with the action. The authors compared the downstream behavior of users following comments that were 5 percentage points above the deletion threshold (the "treatment" in the natural experiment) and those that were 5 percentage points below (the "control"). The researchers found that when a comment was removed in short threads (i.e. threads with 20 or fewer comments), there were fewer rule-breaking comments that followed. A similar effect, however, was not seen in longer threads, presumably because any single comment would be less impactful. They also observed that rule-breakers were less likely to repeat-offend - an effect that lasted for at least four weeks. Importantly, the authors did not see a similar drop in the total number of comments in a thread when a rule-breaking comment was deleted. While users whose comments were deleted posted fewer comments in the subsequent three weeks, by the fourth week they had returned to their previous level of engagement (but not previous level of rule-breaking). The findings of Ribeiro et al (2022) are similar to those in a second study by Srinivasan et al (2019) which also uses "quasi experimental" design. Those researchers look at the removal of rule-breaking comments in the subreddit ChangeMyViews, and they take advantage of the fact that there is often a multi-hour delay before a post is removed. Those authors find that users were less likely to transgress rules after having a post removed compared to "matched" control users who had not yet had their rule-breaking comment removed. It is important to note that ChangeMyViews also provides explanations for why a comment was removed as well as information about how to appeal the removal, and that the subreddit has a three strikes rule, all of which may be contributing factors to the positive impact of comment removals.The similar findings of these two quasi-experiments across different platforms give us confidence that deleting comments which break a forum's rules has a prosocial effect on the behavior of the rule breaker, by making them less likely to break a forum's rules. The findings of Ribiero et al (2022) also suggest that deleting harmful comments has a prosocial effect on other users, reducing the odds they will similarly offend - but, importantly, does not decrease engagement.

Why It Matters

Platforms and online communities usually have an interest to both minimize harm but, at the same time, not reduce engagement. It is therefore promising to see that deleting rule-breaking comments can both have a positive effect on the behavior of users while not measurably decreasing engagement.

Special Considerations

Another auspicious sign is that the positive effects on deleting comments may not depend on whether comments are deleted automatically (as in the Facebook study) or manually by moderators (in the Reddit study). What is important to note, however, is that in both studies the notice of a deleted comment comes with a way for the user to "appeal" the removal decision; in order for comment deletions to have a positive impact on the rule-breaker, then, it may be necessary to include an appeal process (which is consonant with theory on procedural justice and recidivism).

Ribeiro & al. (2022) used the same quasi-experiment methodology to also examine whether "hiding" comments that met a lower threshold of potential harm had a prosocial effect. That test had null findings, so there is no evidence that hiding rule-breaking comments has a positive impact on user behavior.


This intervention entry currently lacks photographic evidence (screencaps, &c.)


Automated Content Moderation Increases Adherence to Community Guidelines

Manoel Horta Ribeiro, Justin Cheng, Robert West
October 19, 2022

Content Removal as a Moderation Strategy

Kumar Bhargav Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, Chenhao Tan
Proceedings of the ACM on Human-Computer Interaction
November 7, 2019

Is this intervention missing something?

You can help us! Click here to contribute a citation, example, or correction.

Further reading

Back to Prosocial Design Library