Validating survey instruments

The data from validation can be used to study the consistency of responses between the interview and reinterview.Estimates of simple response variances can be calculated from the validation data.

Approximately 10% of completed interviews are validated every round.

The main goals of the reinterview are to assess the quality of the interview data and to make sure the interview was done properly.

The selected items are items most respondents will have answered in the main interview and represent a range of expected reliability, including items that are likely to agree between interview and reinterview as well as items that may disagree.

Cases are selected to achieve 10% validation of each interviewer's caseload.

Between November 2000 and July 2001, 989 respondents completed validation reinterviews for round 4.

This produced an overall project validation rate of 11.9 percent of completed interviews.

The short telephone questionnaire included a validation component that asked for details about the respondents' original round 4 interviews (e.g., duration, mode) and information on whether or not they were paid for their participation.

The reinterview component involved re-asking questions that were drawn directly from the youth interview.

For instance, in rounds 2 through 4 it was found that the proportion of cases where the interview and reinterview responses did not match (mismatch rate) was quite low for many factual questions such as the type of housing of the respondent, the highest grade attended, or whether the respondent reported having any income.

Some of these differences could be due to differences in question wording or mode and may be affected by the length of the recall period.

When developing a survey to measure a previously unexplored construct (eg, an athletic trainer's attitudes toward appropriate exertional heat stroke treatment), researchers should employ a four-step process: (1) defining constructs and content domain, (2) generating and judging measurement items, (3) designing and conducting studies to develop a scale, and (4) finalizing the scale.

Tags: , ,