An example of a pop-up that says "Your comment breaks Rule 2. You've been restricted for 24 hours."

Removing Rule-breaking Comments

Reduce rule-breaking and recidivism

Our confidence rating

Validated

Share This Intervention

What It Is

Deleting a comment that breaks a forum's rules

Civic Signal Being Amplified

Welcome
:
Ensure user safety

When To Use It

Reactive

Proactive interventions are designed to shape the overall user experience and environment. They aim to prevent antisocial behavior in the first place, and to foster a positive community.

Interactive interventions engage users in a dialogue or activity to promote prosocial behavior. They encourage reflection, empathy, or positive interactions.

Reactive interventions are triggered by specific user actions or content. They attempt to mitigate potential negative consequences.

If you are a platform or online community and you want to not only remove harmful content but also reduce rule-breaking.

What Is Its Intended Impact

To reduce the likelihood a rule-breaker will be a repeat offender and to reduce rule-breaking among other users.

Evidence That It Works

Evidence That It Works

Ribeiro et al. (2022) used a quasi-experiment design to determine the effect of deleting comments that broke Facebook's rules regarding the use of violent language. At the time of the study, Facebook used an algorithm to detect how likely a comment included violent language and automatically removed comments that passed a given threshold; comments below that threshold were not deleted but instead "hidden" in a thread so that other users needed to click to see them. Users whose comments were removed were notified that their comment broke a rule and was deleted, were told of the consequence (either a warning or 24 hour suspension) and given an opportunity to disagree with the action. The authors compared the downstream behavior of users following comments that were within 5 percentage points above the deletion threshold (the "treatment" in the quasi-experiment) and those that were within 5 percentage points below (the "control").

The researchers found that when a comment was removed in short threads (i.e. threads with 20 or fewer comments), there were fewer rule-breaking comments that followed. A similar effect, however, was not seen in longer threads, presumably because any single comment would be less impactful. They also observed that rule-breakers were less likely to repeat-offend - an effect that lasted for at least four weeks. Importantly, the authors did not see a similar drop in the total number of comments in a thread when a rule-breaking comment was deleted. While users whose comments were deleted posted fewer comments in the subsequent three weeks, by the fourth week they had returned to their previous level of engagement (but not previous level of  rule-breaking).

These findings  are similar to those in another study by Srinivasan et al. (2019) which also used a quasi-experiment design. The researchers looked at the removal of rule-breaking comments in the subreddit ChangeMyView, and they took advantage of the fact that there was often a multi-hour delay before a post was removed. Those authors found that users were less likely to transgress rules after having a post removed compared to "matched" control users who had not yet had their rule-breaking comment removed. It is important to note that ChangeMyView also provides explanations for why a comment was removed as well as information about how to appeal the removal, and that the subreddit has a three strikes rule, all of which may be contributing factors to the positive impact of comment removals.

The similar findings of these two quasi-experiments across different platforms give us confidence that deleting comments which break a forum's rules has a prosocial effect on the behavior of the rule breaker, by making them less likely to break a forum's rules. The findings of Ribiero et al. (2022) also suggest that deleting harmful comments has a prosocial effect on other users, reducing the odds they will similarly offend - yet, importantly, not decreasing engagement.

Why It Matters

Platforms and online communities usually have an interest to both minimize harm but, at the same time, not reduce engagement. It is therefore promising to see that deleting rule-breaking comments can both have a positive effect on the behavior of users while not measurably decreasing engagement.

Special Considerations

Another auspicious sign is that the positive effects of deleting comments may not depend on whether comments are deleted automatically (as in the Facebook study) or manually by moderators (as in the Reddit study). What is important to note, however, is that in both studies the notice of a deleted comment comes with an explanation for the removal and a way for the user to "appeal" the removal decision; in order for comment deletions to have a positive impact on the rule-breaker, then, it may be necessary to include an appeal process (which is consonant with theory on procedural justice and recidivism). See also: Removal explanations.

Ribeiro et al. (2022) used the same quasi-experiment methodology to also examine whether "hiding" comments that met a lower threshold of potential harm had a prosocial effect. Their test had null findings, so there is no evidence that hiding rule-breaking comments has a positive impact on user behavior.

Examples

This intervention entry currently lacks photographic evidence (screencaps, &c.)

Citations

Automated Content Moderation Increases Adherence to Community Guidelines

Manoel Horta Ribeiro, Justin Cheng, Robert West
ArXiV
October 19, 2022
10.48550/arXiv.2210.10454

Content Removal as a Moderation Strategy

Kumar Bhargav Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, Chenhao Tan
Proceedings of the ACM on Human-Computer Interaction
November 7, 2019
10.1145/3359265

Is this intervention missing something?

You can help us! Click here to contribute a citation, example, or correction.

Further reading

Back to Prosocial Design Library