Report An Issue
Thank you!
Your report has been received and we will look into it shortly.
Oh dear. It appears something went wrong while you were submitting the report about something wrong... This is a dilly of a pickle.
A mockup of a prompt telling a user that "Your Comment Is Likely To Be Hurtful To Others" with an option to either edit the post or post it anyway.

Tested

Evoke User's Visibility

Preliminary Flagging Before Posting

Reduce online harrassment

Overview

What It Is

This is an AI-powered intervention, most often Jigsaw's Perspective API, the rates the toxicity of a comment. Typically any comment receiving a high toxicity score will prompt something suggesting they revise their post.

What It Does

This intervention reduces the number of toxic comments posted. It is particularly geared towards "road rage" style comments, in which an otherwise genuine user is immediately in the heat of the moment.

It is worth noting that any API that rates the toxicity of comments is human-written. It will, naturally, carry the perspective and biases of its creators.

When To Use It

This can be used in conjunction with any comment section or platform that primarily relies on short-text posts.

How We Know It Works

In one experiment, researchers at the for-profit social engagement platform, OpenWeb, presented users, who had made a toxic post, a prompt suggesting that they revise their post. The study found that about half users presented with the prompt either revise their comment or decide not to post it when prompted that their comment may be inflammatory.

"The overall positive outcome of this experiment reinforces our belief that quality spaces and civility cues drive a better, healthier conversation experience." writer the study's authors. "A gentle nudge can steer the conversation in the right direction and provide an opportunity for users with good intentions to participate. The feature provides more transparency and education throughout the user engagement journey boosting loyalty and overall community health."

Important Note: this was a study of their own platform, which they sell as a service.

Why It Matters

The findings could help explain that, while a minority of the edits were either trying to trick the system or redirecting their angrily to the prompt itself; most, but not all, of the edits in response to this prompt are done in good faith.

When it comes to moderation technologies there is no one size fits all. We believe this data analysis has also helped us understand and detect online trolls faster and better. If a user is repeatedly ignoring nudges and trying to trick the system, it warrants stronger tools such as auto suspension.

Ido Goldberg, et al.

Examples

This entry is currently being researched & evaluated!

You could help us improve this entry by contributing facts, media, screenshots, and other evidence of its existence in the wild.
Email us

Our Confidence Rating

Tentative

The research supporting this intervention, taken as a whole, is either of corporate origin, or is predominantly limited to descriptive studies. We still list these interventions to encourage further testing, as we believe their merits are promising, and the mechanisms behind them are plausible.

In The Wild

This intervention has precedence, and exists, or has at one time or another, existed in the wild.

Citations

Corporate Origin

Interventional

Altmetric score

September 21, 2020

OpenWeb tests the impact of “nudges” in online discussions

Ido Goldberg, Guy Simon, Kusuma Thimmaiah

Supplemental materials

STUDY DESIGN

Interventional

Interventional studies cover experiments and quasi-experiments. In experiments, individuals meeting eligibility requirements are randomly assigned into an experimental group or a control group. The experimental intervention (protocol, method or treatment) and its alternative(s) are clearly defined and their implementation is closely managed by the researcher. Quasi-experimental studies are empirical studies in which subjects are assigned an intervention, by a non-random method. The researcher may define and manage the alternatives, which could be treatment and control, or two or more different interventions.

The photo with the interlocking yellow-green-blue icon is from the MFA Thesis Project of School of Visual Arts graduate, Kate Styer. You can learn more about her work at: https://www.katestyer.com/keeper

There's more to learn about this intervention. Want to help?

Do you think this intervention could have more benefits, unacknowledged drawbacks, or other inaccuracies that we've neglected to mention here?

We always welcome more evidence and rigorous research to back up, debunk, or augment what we know.

If you want to be a part of that effort, we'd love to have your help!

Email us

Further Reading

Back to Prosocial Design Library
Donate
Contact
About
Library
Home