Email updates

Keep up to date with the latest news and content from IJBNPA and BioMed Central.

Open Access Short paper

Does social desirability compromise self-reports of physical activity in web-based research?

Rik Crutzen1* and Anja S Göritz2

Author Affiliations

1 CAPHRI, Maastricht University, The Netherlands

2 Work, Industrial & Organizational Psychology, University of Würzburg, Germany

For all author emails, please log on.

International Journal of Behavioral Nutrition and Physical Activity 2011, 8:31  doi:10.1186/1479-5868-8-31

The electronic version of this article is the complete one and can be found online at: http://www.ijbnpa.org/content/8/1/31


Received:23 December 2010
Accepted:14 April 2011
Published:14 April 2011

© 2011 Crutzen and Göritz; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

This study investigated the relation between social desirability and self-reported physical activity in web-based research.

Findings

A longitudinal study (N = 5,495, 54% women) was conducted on a representative sample of the Dutch population using the Marlowe-Crowne Scale as social desirability measure and the short form of the International Physical Activity Questionnaire. Social desirability was not associated with self-reported physical activity (in MET-minutes/week), nor with its sub-behaviors (i.e., walking, moderate-intensity activity, vigorous-intensity activity, and sedentary behavior). Socio-demographics (i.e., age, sex, income, and education) did not moderate the effect of social desirability on self-reported physical activity and its sub-behaviors.

Conclusions

This study does not throw doubt on the usefulness of the Internet as a medium to collect self-reports on physical activity.

Findings

This study illuminates the relation between social desirability and self-reported physical activity (PA) in web-based research. Self-report measures are a common way of gathering data in research on PA, because self-reports require fewer financial and logistical resources compared to more traditional methods such as accelerometers. Furthermore, interventions are nowadays increasingly delivered through the Internet [1,2], because of accessibility, convenience, and anonymity. These interventions are routinely accompanied by measurements of PA. In view of the above, the Internet might be a useful environment to collect self-reports on PA. Some researchers, however, state that the social distance [3] and the impersonal nature of the Internet might inhibit trust development [4].

Social desirability - the tendency of respondents to distort self-reports in a favorable direction [5] - might compromise self-reports on PA. Research using latent state-trait models, which account for systematic effects of the situation in which measurement takes place, indicates that the largest proportion of variance in self-reports is attributable to differences in the social desirability trait as opposed to situation-specific conditions eliciting socially desirable self-reporting [6]. If self-reports are indeed influenced by social desirability, taking into account variance due to social desirability may remove some of the error due to the use of self-reports and therewith improve self-reports' validity [7]. Previous research found minimal evidence of an influence of social desirability on scores from two self-report measures of PA in young adults [8]. This research, however, was not web-based, thereby ignoring the social distance and impersonal nature of the Internet. Mode comparison studies (i.e., in which web-based assessment is compared with, for example, paper-and-pencil assessment [9,10] or with telephone interviewing [11]) have been conducted, but the focus of these studies was not on the influence of social desirability itself: These studies did not investigate whether differences in social desirability resulted in differences in self-reported PA. If social desirability is found out to cause distortion in web-based research, this would raise concerns about the validity of web-based research on PA. Therefore, we investigated the association between social desirability and web-based self-reported PA. We put forward the research question:

To what extent is social desirability associated with self-reported PA in web-based research?

Because of the social distance [3] and the impersonal nature of the Internet [4] and in line with previous findings on web-based research on substance use [12], we did not expect social desirability to bias web-based self-reports of PA. In addition, we investigated potential moderating effects of socio-demographics (i.e., age, sex, personal net monthly income in Euros [13], and level of education according to the definitions of Statistics Netherlands) on the effects of social desirability on self-reported PA. In line with a meta-analysis about social desirability distortion [14], we did not expect any moderating effects of socio-demographics.

A longitudinal study was conducted to investigate the relation between social desirability and self-reported PA in web-based research. Data were collected through the LISS panel (Longitudinal Internet Studies for the Social sciences; http://www.lissdata.nl webcite). The reference population for the LISS panel is the Dutch speaking population permanently residing in the Netherlands. In co-operation with Statistics Netherlands addresses were drawn randomly from the nationwide address frame and potential panel members were contacted via post. The sample from the population registers includes individuals who do not have Internet access. These individuals were provided equipment to access the Internet via a broadband connection. Sample members with small-band Internet access were provided with broadband [15].

Social desirability measurements were obtained between May 2008 and August 2008 (T1). In total, 8,722 panel members were invited. Of those, 6,766 completed the social desirability measure (77.6%) - the Marlowe-Crowne Scale [16], which has been validated previously [17]. A high score indicates a large tendency to provide socially desirable responses.

This initial sample of 6,766 panel members was re-invited between November 2008 and December 2008 (T2) to complete the follow-up measures on PA as measured by the short form of the International Physical Activity Questionnaire (IPAQ) [18]. Of those invited, 5,495 completed the IPAQ (81.2%) and were included in the analyses (Table 1). Guidelines for data processing and analysis of the IPAQ (http://www.ipaq.ki.se/scoring.pdf webcite) were adhered to.

Table 1. Sample characteristics (N = 5,495)

Retainees across the two time points did not differ in sex (χ2(1, N = 6,603) = .23, p = .64), income (t(6,285) = .72, p = .47), and education (χ2(5, N = 6,603) = 10.24, p = .07) from panel members who dropped-out. Those who dropped-out, however, were younger than those who completed both the Marlowe-Crowne Scale and the IPAQ (42.1 versus 47.1 years, t(6,601) = 9.16, p < .01).

Multiple regression analyses revealed that social desirability was not associated with total PA (in MET-minutes/week), nor with its sub-behaviors (i.e., walking, moderate-intensity activity, vigorous-intensity activity, and sedentary behavior); both in terms of significance and effect sizes [19,20] (Table 2). Variance due to age, sex, personal net monthly income, and education was taken into account by including these four variables as predictors in the models. Interaction terms between socio-demographics (i.e., age, sex, income, and education) and social desirability were left out of the final models, because none of them were statistically significant. Thus, socio-demographics did not moderate the effect of social desirability on total PA or its sub-behaviors.

Table 2. Effect of social desirability on self-reported physical activity

In line with our hypothesis, this longitudinal study revealed no significant associations between social desirability and self-reported PA in web-based research. Moreover, in agreement with a meta-analysis on social desirability distortion [14], socio-demographics did not moderate the relationship between social desirability and self-reported PA. A possible explanation for the lack of a noteworthy association between social desirability and self-reported PA is that the online setting increases respondents' perceived privacy. Therefore, social desirability might have exerted less influence in this web-based study in comparison with, for example, disclosure in front of an interviewer [21]. Another fact might have contributed to not finding an association between social desirability and self-reported PA: We examined a population-representative sample from an existing panel that was not focused on PA or health as opposed to measuring PA in the context of an intervention aimed at increasing PA. In the context of an intervention aimed at increasing PA the felt pressure to overstate one's PA is likely to be higher than in a non-intervention study, implying that social desirability might distort self-reports of PA in an intervention study after all. This is a subject for future research.

Although social desirability was not found to be related to self-reported PA in this web-based research, this does not imply that self-report measures reveal the same results as direct measures (e.g., accelerometers). A recent review found correlations between self-report and direct measures of PA to be low to moderate [22]. This being only a general caution as this work was not about the comparative validity of self-reports versus direct measures.

Three final points need to be made: (1) Social desirability bias is not the only source of measurement error. Recall error, for example, may also lead to measurement error as may question format [23]. (2) People who dropped-out of the study were younger than retainees. First, a certain level of drop-out is ubiquitous in longitudinal research, also on the web [24]. Second, the dropout in these studies seems to be innocuous, because socio-demographics did not moderate the impact of social desirability on self-reported health risk behaviors. (3) These studies failed to find meaningful associations between social desirability and self-reported PA. Because an absence of evidence of an association does not equal evidence of absence of an association, future research is not precluded from revealing such an association after all. However, the longitudinal nature of this study (e.g., measurements are unlikely to be distorted by participants' unintentional and intentional attempts at portraying themselves as consistent) and the reliance on a large and representative sample gives us confidence in the robustness of our results. This study, therefore, does not throw doubt on the usefulness of the Internet as a medium of obtaining self-reports of PA.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

RC designed the study and AG substantially contributed to the interpretation of the data. RC drafted the manuscript and AG substantially contributed to revising it. Both authors approved the final version of the manuscript.

Acknowledgements

This paper draws on data of the LISS panel of CentERdata.

References

  1. Webb TL, Joseph J, Yardley L, Michie S: Using the Internet to promote health behavior change: a meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy.

    J Med Internet Res 2010, 12(1):e4. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Bock BC, Graham AL, Whiteley JA, Stoddard JL: A review of web-assisted tobacco interventions (WATIs).

    J Med Internet Res 2008, 10(5):e39. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  3. Newman JC, Des Jarlais DC, Turner CF, Gribble J, Cooley P, Paone D: The differential effects of face-to-face and computer interview modes.

    Am J Public Health 2002, 92:294-297. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  4. Joinson AN: Knowing me, knowing you: reciprocal self-disclosure in Internet-based surveys.

    Cyberpsychol Behav 2001, 4:587-591. PubMed Abstract | Publisher Full Text OpenURL

  5. Paulhus DL: Measurement and control of response bias. In Measures of personality and social psychological attitudes. Edited by Robinson JP, Shaver PR, Wrightsman LS. San Diego: Academic press; 1991:17-59. OpenURL

  6. Schmitt MJ, Steyer R: A latent state-trait model (not only) for social desirability.

    Pers Indiv Differ 1993, 14:519-529. Publisher Full Text OpenURL

  7. Jago R, Baranowski T, Baranowski JC, Cullen KW, Thompson DI: Social desirability is associated with some physical activity, psychosocial variables and sedentary behavior but not self-reported physical activity among adolescent males.

    Health Educ Res 2007, 22:3. OpenURL

  8. Motl RW, McAuley E, DiStefano C: Is social desirability associated with self-reported physical activity?

    Prev Med 2005, 40:735-739. PubMed Abstract | Publisher Full Text OpenURL

  9. Ritter P, Lorig K, Laurent D, Matthews K: Internet versus mailed questionnaires: a randomized comparison.

    J Med Internet Res 2004, 6(3):e29. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  10. Wu RC, Thorpe K, Ross H, Micevski V, Marquez C, Straus SE: Comparing administration of questionnaires via the Internet to pen-and-paper in patients with heart failure: randomized controlled trial.

    J Med Internet Res 2009, 11(1):e3. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  11. Nagelhout GE, Willemsen MC, Thompson ME, Fong GT, Van den Putte B, De Vries H: Is web interviewing a good alternative to telephone interviewing? Findings from the International Tobacco Control (ITC) Netherlands Survey.

    BMC Public Health 2010, 10:351. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  12. Crutzen R, Göritz AS: Social desirability and self-reported health risk behaviors in web-based research: three longitudinal studies.

    BMC Public Health 2010, 10:720. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  13. Imputation of income in household questionnaire LISS panel [http://www.lissdata.nl//dataarchive/hosted_files/download/24] webcite

  14. Richman WL, Kiesler S, Weisband S, Drasgow F: A meta-analytic study of social desirability distortion in computer-administered questionnaires, tradition questionnaires, and interviews.

    J Appl Psychol 1999, 84(5):754-775. Publisher Full Text OpenURL

  15. Start of the LISS panel: sample and recruitment of a probability-based Internet panel [http://www.lissdata.nl/assets/uploaded/Sample%20and%20Recruitment_1.pdf] webcite

  16. Crowne DP, Marlowe D: A new scale of social desirability independent of psychopathology.

    Journal of Consulting Psychology 1960, 24:349-354. PubMed Abstract | Publisher Full Text OpenURL

  17. Fischer DG, Fick C: Further validation of three short forms of the Marlowe-Crowne Scale of Social Desirability.

    Psychol Rep 1989, 65:595-600. OpenURL

  18. Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, Pratt M, Ekelund U, Yngve A, Sallis JF, et al.: International physical activity questionnaire: 12-country reliability and validity.

    Med Sci Sport Exer 2003, 35(8):1381-1395. Publisher Full Text OpenURL

  19. Crutzen R: A systematic review on computer-based education for patients with hypertension: what about effect sizes?

    Health Educ J 2010, 69:365-366. Publisher Full Text OpenURL

  20. Crutzen R: Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers.

    Int J Behav Nutr Phy 2010, 7:29. BioMed Central Full Text OpenURL

  21. Tourangeau R, Smith TW: Asking sensitive questions: the impact of data collection mode, question format, and question context.

    Public Opin Quart 1996, 60:275-304. Publisher Full Text OpenURL

  22. Prince SA, Adamo KB, Hamel ME, Hardt J, Connor Gorber S, Tremblay M: A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review.

    Int J Behav Nutr Phy 2008, 5:56. BioMed Central Full Text OpenURL

  23. Gmel G, Lokosha O: Self-reported frequency of drinking assessed with a closed- or open-ended question format: a split-sample study in Switzerland.

    J Stud Alcohol Drugs 2000, 61:450-454. OpenURL

  24. Göritz AS: The long-term effect of material incentives on participation in online panels.

    Field Method 2008, 20:211-225. OpenURL