PDN & Jigsaw convening: Accelerating independent research

On December 1st, twenty five researchers joined Prosocial Design Network and Jigsaw for a day-long convening to tackle the question "how can we collectively accelerate independent, ecologically valid research around prosocial design?"

It was an incredible day of knowledge-sharing about promising approaches to conducting independent research, envisioning collaborative research projects and planning how researchers can work together to fuel more prosocial design research.

The seed idea for the event came from our library team, which reviews research to identify evidence-based prosocial design patterns. The vast majority of studies we see use survey- and other lab-based experiments, which are informative but also are low on what social scientists call "ecological validity", i.e. they can show a design intervention works in a lab but not necessarily in real life. Only a few dozen publicly available studies test design patterns on platforms, making them high on ecological validity. (Platforms presumably have tons of ecologically valid studies themselves, but they're not for public consumption.) 

There's a reason there's so little independent, ecologically valid research: it's hard to do. That's why we brought together researchers who are taking promising approaches to study the effectiveness of prosocial digital design - to see if we could pull our collective minds together to imagine innovations, including methodologies and external supports, to fuel more ecologically valid research. 

The day began with talks from Amy Bruckman, Dominic DiFranzo, Max Allamong and Kylan Rutherford who each presented promising approaches to studying prosocial design in ecologically valid settings, including qualitative studies, simulated social media environments, LLM testbeds and browser extension studies. Julia Kamin wrapped up the talks with a brief tour of other successful methodologies we've seen at PDN.

The bulk of the day was made up of breakout groups where participants - including nineteen academics, six researchers from industry and society and six Jigsaw-ers (see below)- mapped the merits and challenges of research approaches, applied those methodologies to potential research projects and imagined innovations to accelerate ecologically valid research. 

The amount of knowledge sharing, inspiration and ideation that came out of the day was overwhelming. Over the next couple of weeks, we'll be synthesizing the output from the convening and, more importantly, charting next steps to convert the inspiration from the event into action with three sets of working groups to a) create a guide for researchers on how to conduct ecologically valid research, b) promote models for collaborative research with platforms that maintains research integrity and c) build infrastructure and supports to facilitate independent research.

While a full synthesis is bit away, a few strong themes that emerged from the gathering:

• Researchers were excited to learn about the potential of both new and well-established methods to study prosocial design - and to see others taking similar approaches to their work.
• Researchers also noted the advantages that can come from continuing to keep up to date with advances, as well as sharing best practices and resources, for example creating a central repository of code.
• There is likewise potential to pool resources and efforts to accelerate the use of ecologically valid research approaches in testing prosocial design.

We're both daunted and excited to see how the energy from the gathering can be transformed into action. If you're a researcher, technologist or civic organization who wants to help us make that happen, reach out to julia@prosocialdesign.org.

Finally, the event couldn’t have happened without the generous support of Civic Health Project and USC Marshall’s Neely Center for Ethical Leadership and Decision Making as well as all the event’s incredible organizers and participants: