How well do we really understand social media?

Rebecca Dolgin, Simone Calbi, Jeffrey Huang, & Michael F. Schober

Twitter is the social media platform people use most often to share opinions (Pew Research Center, 2021). Yet, the vast majority of content on Twitter isn’t original, it’s retweets—the online equivalent of a head nod, high five, or scowl. All of this agreement or disagreement makes a tacit assumption that the tweet has been understood. We hypothesize, however, that misunderstandings on Twitter may be prevalent and that they go undetected or repaired.

Prior evidence about online comprehension suggests there may be more variability of interpretation than one might assume (Kern et al., 2024; Beck et al., 2022; Prabhakaran et al., 2021; Fox Tree et al., 2020; Kellner & Schober, 2018; Anand et al., 2011). Using a novel method, this study verified the poster’s intentions directly, as opposed to inferring the poster’s general intentions or relying on annotators’ judgments. To do this, we collected a corpus of tweets posted during a specific time period and that included “Roe v. Wade”. From this we solicited authors and asked about the stance they intended to express in the tweet, as well as other questions about their political ideology and their demographic characteristics.

We then asked a new set of participants (N = 101) a series of questions about each tweet and additional questions about their Twitter usage, political ideology, and demographics. Despite all posters being extremely confident (90-100%) that their tweet would be understood as intended, readers’ alignment with the authors’ self-reported stance in each tweet ranged from as low as 48.5% and no higher than 89.1% of readers.

Participants’ age significantly predicted the degree of alignment, with younger participants more likely to align on more of the six tweets. Further analyses including more factors suggest that the combination of age and political ideology (but not Twitter use) significantly predicts alignment (R2 = .17, F(2, 96) = 9.52, p < .001; adjusted R2 = .15).