The Relationship Between Mental Health and Substance Abuse Among Adolescents

APPENDIX B: TECHNICAL APPENDIX

This appendix discusses sample weighting, statistical tests, and suppression of estimates due to low precision. This report presents estimates of the percents of persons aged 12 to 17 in the total NHSDA target population and in age and gender groups who used substances, were dependent, or needed illicit drug abuse treatment during specified time periods preceding the interview. The estimates of this report are based on sample survey data rather than on complete data for the population. This implies that the estimates must be weighted to take into account the complex sample design of the NHSDA. All estimates presented in this report are weighted to take into account the complex sampling design of the NHSDA. The basic sampling weights of each annual NHSDA are equal to the inverses of the probabilities of selection of sample respondents. In other words, the smaller a respondent's chance of entering the sample, the larger the weight of that respondent in the calculation of unbiased estimates for the target population. To obtain the final NHSDA weights, the basic sampling weights are adjusted to reflect household-level and individual-level nonresponse and further adjusted to ensure consistency with U.S. census population totals. Thus, the estimates presented in this report are unbiased estimates for the U.S. civilian, noninstitutionalized population aged 12 and older or for specified subgroups of that target population. (For details of NHSDA weighting, see SAMHSA, 1998, Appendix D.)

All comparisons, as well as the individual rates themselves, are subject to sampling error that is readily quantified. Sampling error for an individual rate results from asking questions of a sample rather than of everyone in the surveyed population. Estimates in this report were rounded to the nearest tenth of one percent and tested to confirm that they met a required level of statistical precision. Estimated percents were considered to be of low precision if the standard error of the estimate was greater than 17.5 percent of the log transformation of the estimate. Low precision typically occurs for small subgroups when percents are close to zero or 100, but the large combined sample size of the 1994-B-1995-1996 NHSDAs ensured that few estimates had to be suppressed because of low precision. Estimates that were suppressed because of low precision are noted using asterisks in the text tables of this report.

Sampling theory provides the basis for calculating confidence intervals around the estimates and tests of significance in comparing two estimates. The size of the intervals and the tests of significance depend on: sample size; the interaction between the sampling procedure used and the distribution of a particular variable in the population, that is, the appropriate design effect; and the degree of confidence required in the interval estimate, or level of protection against incorrect inferences, required in the test of significance.

This report presents results of statistical significance tests for comparisons of subgroups defined by gender, age, and severity of emotional or behavioral problems. Differences between groups in prevalence and frequency of substance use, rates of dependence and treatment for drug use were tested for statistical significance using t tests. The ttest takes into account the sizes of the subsamples being compared and the degree of variation among sample members. An observed sample difference is designated as "statistically significant" if the probability of a sample difference equal to or larger than the observed sample difference is less than or equal to a given significance level, usually .05. Because of the large number of comparisons in this report, a more conservative significance level was adopted. Differences discussed in the text were statistically significant at the .001 level or lower.

The appendix tables present standard errors for all estimates that are presented in Chapter 3 of this report. A test statistic can be computed using the estimates reported in any text table and the standard errors reported in the corresponding standard error table of the appendix. Like the estimates themselves, the standard errors of these estimates fully take into account the complex sampling design of the NHSDA. In particular, the variance estimation software (SUDAAN) used the method of Taylor Series linearization to adjust all standard errors for correlations between observations due to the multistage sample design of NHSDA. (See SAMHSA, 1998, Appendix D for details of the NHSDA sample design and variance estimation methodology.)

The standard error tables have the same formats as the text tables for which they provide statistical documentation. The appendix table presenting the standard errors for estimates shown in a specified text table has the same number as the text table except for the prefix "A." For example, appendix table A3.1 presents the standard errors for the estimates show in text table 3.1.

Nonsampling error, which includes nonresponse, misreporting, and miscoding, cannot be measured as satisfactorily as sampling error. A series of studies on the validity and reliability of general population survey data are reported elsewhere (see Turner, Lessler, and Gfroerer, 1992).

Chapter 4 displays the results of logistic regression analyses. Specifically, logistic regression models were used to determine whether specific measures of psychological functioning are associated with measures of substance use. In order to assess correlates of substance use and variables that mediate the relationship between psychological functioning and substance use, we analyze the logistic regression models in Chapter 4 separately for males and females for each of the three age groups. Consistent with recent statistical advice on the use of weights in regression analysis (e.g., Winship and Radbill, 1994), the estimates of Chapter 4 use weighted data, in which estimates are weighted to take into account the complex sample design of the NHSDA, rather than unweighted data. It is important to use weighted data in order that the logistic regression estimates will gauge the average effects of factors affecting adolescent substance use in the NHSDA target population. The precise method of statistical estimation applied in these logistic regression analyses is called "weighted maximum likelihood estimation." The estimation method is implemented in the computer program SUDAAN and discussed in Shah et al. (1995).

The NHSDA is the only survey that regularly produces estimates of drug use among members of the civilian, non-institutionalized population aged 12 and older of the United States. The survey is an appropriate vehicle for estimating prevalence of use for cigarettes, alcohol, and other illicit drugs because it reports use of various substances that does not ordinarily come to the attention of administrative, medical, or correctional authorities. In-person interviews with a large national probability sample are the best way to estimate drug use in the general population of the United States.

Although the NHSDA is useful for many purposes, it has certain limitations. First, the data are based on self-reports of drug use, and their value thus depends on respondents' truthfulness and memory. The validity of self-report drug use data has been established in previous research (see Turner, Lessler, and Gfroerer, 1992). The NHSDA procedures encourage honesty and recall. Nevertheless, some under- and overreporting may have occurred. Second, because the population of the survey is defined as the civilian, noninstitutionalized population of the United States, a small proportion (less than 2 percent) of the population is excluded: those living in institutional group quarters (e.g., prisons, nursing homes, treatment centers), those with no permanent residence (e.g., homeless people), and active military personnel. As a result, estimates of substance use derived from the NHSDA may be slightly lower, in particular the prevalence estimates of rarely used drugs such as heroin and hard-core drug use, as studies have demonstrated that alcohol and illicit drug use in populations living in institutional settings and those with no permanent residence differed significantly from that of the household population (National Institute on Drug Abuse, 1993).

This page was last updated on June 16, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.