Table 1

Study internal and external validity coding criteria

Criteria description

Scoring for criteria


Internal validity criteria


1

Was the method of randomization appropriate?

Y = 1; N = 0


2

Were baseline groups equivalent on important demographic measures?

Y = 1; N = 0; UK = 0


If No, was analysis conducted to estimate/adjust for effect of demographic measure on outcomes?

Y = 1; N = 0; UK = 0


3

Did the design of the study include comparison to a no treatment control group or a group with either no technology or no tailoring?

Y = 1; N = 0


4

Was retention rate ≥ 80% at post-test/post-intervention follow-up?

Y = 1; N = 0


5

Were Outcome Measurement instruments valid? Was there a description of instrument reliability/validity (reference or coefficients) OR did they use a well-established known valid measure?

Y = 1; N = 0


6

Was an objective measure of behavior change used or did they rely solely on self-report measures?

Objective = 1; Self-report only = 0


7

Was power analysis reported to determine sample size?

Y = 1; N = 0


8

Were analyses conducted with consideration for missing data that maintain fidelity of the randomization (e.g. intention to treat, imputation)? Note: if 100% retention then N/A

Y = 1; N = 0


9

Was the intervention based on theory?

Y = 1; N = 0


External validity criteria


1

Were recruitment methods and/or inclusion & exclusion criteria sufficiently described?

Both = 1, Either = 0.5; None = 0


2

Were participation/recruitment rates provided OR Are analyses reported on the similarity and differences between participants versus either those who decline or the intended target audience (individuals or settings)?

Y = 1; N = 0


3

Was a large heterogeneous sample used? Was the representativeness of participants described? Was a homogenous/heterogeneous sample sought for target population? Do the exclusion criteria used reduce the generalizability of findings?

Generalizable population = 1


4

Was the representativeness of the setting described? Was the study conducted in an uncontrolled/controlled setting? Can their findings only be generalized to the limited conditions within which the research was carried out?

Generalizable setting (real-life) = 1; Controlled = 0


5

Were all participants who entered trial accounted for at its conclusion i.e. Are data on attrition by condition reported OR was dropout rate described?

Y = 0.5; N = 0

Are drop-outs' compared to completers OR are the dropout's characteristics and reasons for drop-out described?

Y = 0.5; N = 0


6

Was the use of comparison conditions relevant to real-world decisions? (the computer-tailored treatment group was compared with either non-technology or non-tailored or alternative programs rather than no treatment)

Y = 1; N = 0


7

Are data on the costs presented?

Y = 1; N = 0


8

Was there sufficient description of the intervention, including: method of tailoring, duration and intensity (amount of contact time required)?

Y = 1; N = 0


9

Are data reported on maintenance or longer-term effects?

Short-term = 0; Medium term = 0.5; Long-term = 1


Neville et al. International Journal of Behavioral Nutrition and Physical Activity 2009 6:30   doi:10.1186/1479-5868-6-30

Open Data