The Trouble with Sabatini and Sarracino and the "Growing Evidence" of the Negative Consequences of Social Networking
Sabatini and Sarracino's (2014) paper, "Online networks and subjective well-being," is making the social media and blogosphere rounds because it sounds like it found out a bunch of scary things.
But it has problems as a study. And because journalism about social science is often dodgy at best, we get thin, pithy and generally reckless overclaiming more often than not. Articles on this study do exactly that. Look at paragraph 2 of this article that is making the rounds through my Facebook feed for an example.
So here's the troubles:
1. Overclaimed media effects assumptions
On page 8 they trot out all the technological evils of computers that cause people to be bad online. First, we can be anonymous and so will be aggressive and rude. Second, when we are NOT anonymous we can just block people, thus: aggressive and rude. Convenient! Whatever the nature of the facts, their bias can still be right!
This line of research, that the tech made me do it, was wrong a decade ago, when (academic ego alert!) I published on it :). Now it is just a bad idea. Of course ,they do cite themselves in that section, so I guess that counts for something (But then I can cite myself too, ha ha!)
2. Cherry-picked lit review analysis
On page 3, the authors string cite article on both sides of issues of trust and well-being (two very different things clumped into a hodgepodge here). The answer is left up in the air. In the later lit review, where they return to the issue, they use 5 studies (out of ALL the reams of data out there on social networking) to conclude that it is bad, without actually ever really considering all the pro-well-being articles they glossed over before. This is fishy.
3. Correlation is tricky
Page 19, they cite work indicating that television viewing is correlated with lack of life satisfaction. But whereas earlier in the paper they spend some time trying to untangle variables and causality, here they just kind of leap forward. Is TV depressing or is TV what we do when we are depressed? And if you think reality is complex enough that both of those could be true in different ways, in different degrees, for different people, well then this kind of study on a big giant poll is not for you. Good studies published in good journals generally are aware of the pitfalls here in the way that this study is not. Here's what Pea et al. (2012) say would be required before making the conclusions Sabatini and Sarracino leap to:
We express cautions similar to that issued by Rideout et al. (2010) in their study of media in the lives of 8- to 18-year-olds: “This study cannot establish whether there is a cause and effect relationship between media use and [social consequences]. And if there are such relationships, they could well run in both directions simultaneously” (p. 13). The kinds of empirical studies that could provide warranted inferences about causal relationships between media use patterns, face-to-face interaction, and social well-being would need to be longitudinal, to follow specific cohorts, and to provide either experimental interventions with controls or “natural experiments” that allow for controls. In either case, groups matched according to characteristics of participants presumed to make a difference would be compared with respect to their different experiences with media use. Then outcomes in terms of social well-being, sleep, and other variables would be compared for experimental group participants and controls. It would also be valuable in future studies to include new survey items that distinguish media use and multitasking associated with media production rather than consumption, as production activities are more likely to be positively associated with the development of educationally valued technological fluencies ( Barron, Walter, Martin, & Schatz, 2010), which might result in healthier social development. We also need to have a more differentiated account of the content and purposes of media use and media multitasking than our study provided, as some forms of individual and social learning uses of media and media multitasking (as in parents' “active mediation” in video co-viewing with children: Reiser, Williamson, & Suzuki, 1988) could contribute positively to social and cognitive development. New studies could also ground claims about children's media use with more granular methodologies such as media time-use diaries, experience-sampling methodologies, electronic monitoring techniques, wearable computers for media capture, or direct observations (also see Vandewater & Lee, 2009). [p. 335]
Yeah, that's not happening here.
4. Sampling is hard
The authors critique other studies which contradict their results for being of limited samples. Specifically they are tired of reading about samples of American college students. Fair enough. So college students like Twitter. Okay. Perhaps their liking of it is part of its positive impact on their lives? Perhaps segmenting the giant Italian data set into age would reveal something more complex here? They spend pages statistically correcting for broadband access (?!). But nothing on demographics? I'd like to see them segment samples of the survey data (if possible) before they start in on others' samples. And, really, the kind of sample gathered in studies like this seems relevant and helpful.
5. Advocacy should be based on, you know, facts, maybe?
They admit their results might be "spurious" on page 26 but then just go merrily on to suggest that Facebook should change its settings as a result of their work. Wait. Really?
6. It helps to have perspective
They consistently cite their own previous work when faced with competing ideas. Echo chambers are rarely good places for ideas to grow. And this leads them to flat-out misread other articles. They rely quite a bit on Helliwell and Huang (2013) when the going gets tough. But H & H, in addition to doing a much more thorough statistical analysis and doing some demographic breakdown of their data, as well, conclude the following about social networks and happiness: "Second, we find that comparably measured networks of on-line friends have zero or negative correlations with subjective well-being, whether or not allowance is made for the influence of other factors." Hmm. S & S characterize this result as "These results are in line with Sabatini and Sarracino (2014), who found that participation in SNS might destroy social trust, and with Helliwell and Huang (2013), who found that face-to-face interactions are positively associated with happiness, while online networks are not." (p. 26). Wait a minute. H & H found that happiness was not correlated with the SIZE of the SNS network. That means more Facebook friends does not make you happier than only a few, and it might even make it worse. Well, duh.
The lesson in all of this? Be careful of scary stories on scary social media studies. This work is hard to do, and there are few certainties. But you knew that already, right?
