Risk & Protective Factors for Adolescent Drug Use:  1999 NHSDA

Appendix D: Methodological Changes Between the 1997 and 1999 NHSDAs

As mentioned in Chapter 1, the National Household Survey on Drug Abuse (NHSDA) was administered using computer-assisted interviewing (CAI) methods for the first time in 1999. To assess the impact of this change in data collection mode, the 1999 survey utilized a dual-sample design. In addition to the 66,706 individuals interviewed using the CAI method, 13,809 individuals completed the same questionnaire using a paper-and-pencil interviewing (PAPI) methodology. Because of this methodological consistency with previous years of the survey, the 1999 PAPI sample was designed to be the main basis for relating 1999 drug use estimates to estimates from 1998 and prior years. Even though it was implemented within the sampling frame for the 50-State sample designed for the CAI survey, an extra stage of sampling and a within-household screening procedure were added to allow coordinated oversampling of the Hispanic and black households in a manner comparable with the 1998 and prior years' surveys. Weighting, editing, and imputation procedures were also conducted in a manner comparable with prior years' surveys.

In spite of the efforts taken to maintain total methodological comparability, analyses have suggested that the 1999 PAPI data may not be comparable with earlier data. Investigations into possible technical problems related to data collection, response rates, Quarter 1 start-up problems, weighting, and editing and imputation were conducted to see whether any procedural changes or errors may underlie the problem. Although no technical problems or obvious causes associated with these factors have been discovered, one line of inquiry within this general investigation was to investigate possible interviewer experience effects.

It was discovered that respondents were more likely to report substance use in interviews conducted by inexperienced interviewers than with experienced interviewers. This was exhibited in a small difference in predicted prevalence rates based on prior NHSDA experience and a continuing small, but often statistically significant decline in predicted prevalence rates as interviewers accumulated experience during the year. Under continuing operations with about the same level of effort from year to year, the experience of interviewers would be approximately matched for 2 succeeding years causing both years' estimates to be influenced in comparable ways. Because of the expansion of the sample in 1999, the interviewers in 1999 were generally less experienced than in prior years. Analytical studies that took account of the differences in interviewer experience distributions showed that under comparable conditions, the 1999 estimates would be lower than shown by the direct estimates. Initial analysis of the CAI sample indicates much smaller interviewer experience effects. This tends to validate the decision to move to the CAI technology as a means of reducing survey errors associated with the interviewing environment.

The prevalence of past year marijuana use among youths from the 1997 PAPI, the 1999 CAI, the 1999 PAPI, and the 1999 PAPI after adjusting for field interviewer experience are presented in Exhibit 5.1 in Chapter 5. A comparison of the 1997 PAPI and comparable 1999 PAPI (adjusted) estimates clearly shows that the prevalence rate for youth use of marijuana declined over these 2 years. All other comparisons between 1997 and 1999 in Chapter 5 are based on the adjusted 1999 PAPI data.

This page was last updated on July 14, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.