The Right Community Rules? Lessons from Reddit: A Pro-Social Recap

Setting rules in online communities has a positive effect. That much we know with strong confidence. But does it matter what those rules are? Does it vary across communities? And how should a community figure out what rules are best for them?

Galen Weld, a researcher at the University of Washington and soon-to-be research scientist at the Wikimedia Foundation, has conducted several ambitious studies looking across thousands of subreddits that begin to point toward some answers. His research on community values and perceptions of moderators with Amy X. Zhang, Tim Althoff and Leon Leibmann combines large-scale data analysis with direct surveys of community members, offering insights into what communities value, how they structure their rules, and which approaches actually lead to better governance outcomes.

In this Pro-Social conversation, Galen joined us to discuss his research and share thoughts on how to support moderators in making better decisions for their communities.

Different Communities, Different Values

Galen started by explaining why Reddit makes such a compelling research site. The platform is composed of thousands of subreddits, each with its own rules, moderators, and topics. This creates a natural laboratory for studying governance, with existing diversity in values and practices across communities.

Galen et al. conducted the largest set of surveys asking community members about their values on Reddit, gathering responses from about 3,000 participants across about 5,000 different communities. They asked about nine different values including content quality, diversity, trust and safety, inclusion, and democratic practices.

The findings revealed both commonalities and important differences. Across communities, people consistently emphasized content quality, membership diversity, and trustworthiness. But specific types of communities showed distinct value priorities. News-sharing communities considered trust and content quality particularly important. Identity-focused communities like LGBTQ groups or spaces for young people placed relatively more emphasis on inclusion, diversity, and safety.

The key takeaway: there isn't a one-size-fits-all set of values. Different communities have different needs and may want to make different tradeoffs in how they structure themselves.

An especially revealing finding came from looking at disagreement within communities. While community members generally agreed in the abstract that their community should be safe, they disagreed considerably about how safe the community actually is right now. Galen's hypothesis is that people who have personally experienced safety issues perceive the community very differently than those who haven't, and in many communities, those who feel unsafe may be in the minority. This highlights why governance practices need to protect minority voices, whose concerns are just as important even if they're not held by the majority.

A Taxonomy of Rules

Galen and his team also pulled the text of rules from about 10,000 subreddits and analyzed how those rules differ across communities.

They looked at several dimensions. One was how rules are phrased: prescriptively, telling people what to do ("be nice"), versus restrictively, telling people what not to do ("don't be mean"). They also examined what aspects of participation rules target, whether content, formatting, or the people themselves. And they identified specific common rule types like those addressing spam, buying and selling, or linking to particular sources.

As with their values survey, some patterns emerged. Spam and low-quality content rules are particularly common, as are rules about respecting other community members. Communities generally phrase their rules more restrictively than prescriptively.

But again, differences appeared by community type. Identity communities more commonly have rules about who can and who is welcome to participate. Hobby communities are more likely to have rules about commercial activities. News-sharing communities often restrict which websites can be linked or have specific rules about news sources.

Which Rules Lead to Better Governance?

The most challenging question is: which rules lead to better outcomes? Measuring governance quality is difficult because what does "working well" even mean?

Galen and his fellow researchers developed a clever approach. They created a classification pipeline to detect and analyze cases where people talk about moderators in their communities, expressing positive, neutral, or negative sentiment. Across thousands of communities, they identified when people said things like "the moderators here are great" or, more commonly, less flattering assessments of the  moderators.

By connecting this sentiment data with information about community rules, they could see which governance approaches were associated with more positive discussion.

They found that rules about engaging with other community members were associated with more positive and fewer negative comments about moderators. The same was true for rules about formatting and discoverability mechanisms, and rules about who participates in the community.

The question of how rules are phrased also mattered significantly. Communities with more prescriptive rules saying what you should do tended to have more positive discussion about governance. Communities with more restrictive rules saying what you can't do tended to have more negative discussion. This aligns with offline criminology literature and social psychology research suggesting that framing things positively can lead to better outcomes.

Another interesting finding concerned the effects of adding new rules. When communities add a new rule, there's generally an improvement in perceptions of governance. But this effect isn't durable and wears off after about six months. On average, things get better for a bit after a new rule is added, but after half a year, sentiment isn't much different from before the rule existed.

What's Next: Making Moderation More Data-Driven

Looking forward, Galen said that in many volunteer-run online communities, current moderation and governance practice could be far more data-driven than it is. There's low-hanging fruit in providing communities with additional tools and information to help them make more well-informed decisions.

He highlighted past research by Nathan Matias that worked with communities to run A/B tests on specific interventions like announcement posts or bans and projects like PolicyKit and modgov that allow communities to create constitutions or other formalized governance expressions. The vision is for communities to better understand what's happening in their spaces and to be able to run experiments or enforce specific governance practices based on evidence rather than guesswork.

Watch the full conversation with Galen Weld:

‍

About the Prosocial Design Network

The Prosocial Design Network researches and promotes prosocial design: evidence-based design practices that bring out the best in human nature online. Learn more at prosocialdesign.org.