Risk & Protective Factors for Adolescent Drug Use:  1999 NHSDA

Appendix E: Comparison of Similar But Not Identical Items Included in the 1997 and 1999 NHSDAs

E.1 Introduction

As mentioned in Chapter 5, comparisons made between the risk and protective factors of youth substance use in the 1997 and 1999 National Household Surveys on Drug Abuse (NHSDAs) are limited to factors included in both years of the survey. Chapter 5 includes analyses of change between 1997 and 1999 in 11 questions that were directly comparable between the 2 years. In addition, another set of questions related to risk and protective factors of youth substance use and included in both 1997 and 1999 underwent improvements in the wording or the response options between the 2 years. Because of these alterations to the questions, it is difficult to determine whether observed changes in responses to these questions between 1997 and 1999 can be attributed to real changes in the factors or to the changes in the questions. However, examining the overall trends in the direction of change between 1997 and 1999 in these variables may be instructive in helping to understand the reduction in marijuana use between those 2 years.

This appendix describes a set of questions included in both the 1997 and 1999 NHSDAs that had differences in wording or response options between those 2 years. First, the set of variables is described, including an explanation of how they changed between 1997 and 1999. Next, comparisons are made between the 1997 and 1999 surveys in the distributions of these questions and the associations between the questions and past year marijuana use. Finally, there is a discussion of the amount of variation in youth past year marijuana use that can be accounted for by this set of questions when combined with the set of demographic variables and the 11 questions that were directly comparable between the 2 years of the survey. As was the case in Chapter 5, all comparisons presented in this appendix are between 1997 paper-and-pencil interviewing (PAPI) data and 1999 PAPI data that have been adjusted for field interviewer (FI) experience.

E.2 Similar But Not Identical Questions Included in the 1997 and 1999 NHSDAs

Table E.1 presents seven questions related to risk and protective factors of youth substance use that were similar but not identical between the 1997 and 1999 surveys. These include questions about parental communication about the dangers of substance use, parental attitudes toward youth smoking, friends' use of marijuana, friends' use of alcohol, participation in gang fights in the past year, number of times attended religious services in the past year, andgrades in school in the last completed semester. Differences in the wording or response options between the 2 years of the survey are shown in italics in Table E.1.

For the question on parental communication about the dangers of substance use, youths in 1997 were asked, "During the past 12 months, have you spoken with a parent or other adult about the dangers of alcohol or drug use?" In 1999, youths were asked, "During the past 12 months, have you talked with at least one of your parents about the dangers of tobacco, alcohol, or drug use?" The 1997 question asks about communication with parents or other adults and asks only about communication regarding the dangers of alcohol and drug use. The 1999 question asks only about communication with parents and also includes communication about the dangers of tobacco use.

For the question on parental attitudes toward youth smoking, youths in 1997 were asked, "How would your parents feel if they thought you smoked 1 or more packs of cigarettes per day?" In 1999, youths were asked, "How do you think your parents would feel about you smoking 1 or 2 packs of cigarettes per day?" However, the more notable change was in the response options. In 1997, the response options were (1) Very upset, (2) Somewhat upset, or (3) Not at all upset, whereas in 1999 the response options were (1) Strongly disapprove, (2) Somewhat disapprove, or (3) Neither approve nor disapprove. This change makes comparisons of the two questions difficult because it is possible that youths may have believed that their parents would disapprove of their smoking but would not become upset by it.

For the question on friends' use of marijuana, youths in 1997 were asked, "How many of your close friends have tried marijuana once or twice?" Youths in 1999 were asked, "How many of your friends would you say use marijuana or hashish?" The 1997 question asks whether close friends trying marijuana, which implies occasional use. The 1999 question talks about a broader group of friends (not just close friends) using marijuana or hashish, which implies a more regular usage. The response options in 1997 were (1) None of them, (2) A few of them, or (3) Most of them. In 1999, an additional response option of ("All of them") was added. For the purposes of the comparisons made between these questions in this report, the responses "Most of them" and "All of them" from the 1999 survey were combined.

For the question on friends' use of alcohol, youths in 1997 were asked, "How many of your close friends have had 5 or more drinks once or twice a week?" Youths in 1999 were asked, "How many of your friends would you say get drunk at least once a week?" The 1999 question asks about a broader group of friends (not just close friends), and asks about "getting drunk" (rather than about having five or more drinks) at least once a week (rather than once or twice a week). The response options were the same as for the question on friends' use of marijuana.

For the question on participation in a gang fight, youths in 1997 were asked, "During the past 12 months, how many times have you gotten into a gang fight?" Youths in 1999 were asked "During the past 12 months, how many times have you taken part in a fight where a group of your friends fought against another group?" The response options in 1997 were (1) 0 times, (2) 1 or 2 times, (3) 3 or 4 times, or (4) 5 or more times, whereas in 1999 the response options were (1) 0 times, (2) 1 or 2 times, (3) 3 to 5 times, (4) 6 to 9 times, or (5) 10 or more times. For the purposes of the comparisons made between these questions in this appendix, the responses for both years were coded as (1) 0 times, (2) 1 or 2 times, or (3) 3 or more times.

For the question on attending religious services, youths in both 1997 and 1999 were asked how often they had attended religious services in the past 12 months. The response options in 1997 were (1) Never, (2) About once or twice, (3) About once or twice a month, (4) Attended every week, or (5) Attended several times a week, whereas in 1999 the response options were (1) 0 to 5 times, (2) 6 to 24 times, (3) 25 to 52 times, or (4) More than 52 times. For the purposes of the comparisons made between these questions in this appendix, the responses for both years were coded as (1) About once or twice a month or less (1997) / 0 to 24 times (1999) or (2) Attended once a week or more (1997) / Attended 25 or more times (1999).

For the question on academic performance, youths in both 1997 and 1999 were asked to report their grades for the last semester or grading period that they completed. The response options in 1997 were (1) Mostly A's or B's, (2) Mostly B's or C's, (3) Mostly C's or D's, or (4) Mostly D's or below, whereas in 1999 the response options were (1) A+, A, A- average; (2) B+, B, B- average; (3) C+, C, C- average (1999); or (4) D or less than D average. For the purposes of the comparisons made between these questions in this appendix, the responses for both years were coded as (1) Mostly C's or D's or better (1997) / C+, C, C- average or better (1999), or (2) Mostly D's or below (1997) / D or less than D average (1999).

E.3 Comparison of the Distributions of Factors in 1997 and 1999

Table E.2 presents the percentages of youths who selected each response category in both 1997 and 1999, as well as a test of whether the differences in responses between the 2 years are statistically significant (values of p <.05). There were significant differences between the distributions for each of the seven questions that were similar but not identical between 1997 and 1999. As mentioned earlier, the differences in the wording or response options of these questions between the 1997 and 1999 surveys prevents any meaningful interpretation of these differences for the individual questions.

For example, a higher percentage of youths in 1999 reported that their parents would strongly disapprove if they were to smoke one or more packs of cigarettes a day (89.4 percent)than reported in 1997 that their parents would be very upset if they smoked one or two packs of cigarettes a day (81.2 percent). One explanation for this is that there was a real change in youths' perceptions of how their parents would feel if they were daily smokers. A second explanation is that some youths may have believed that their parents might strongly disapprove of youth smoking without getting very upset about it. Both explanations are consistent with the pattern of change, so neither can be ruled out as the cause of the change.

Despite the difficulty in interpreting the reason for the changes for each individual question, the overall pattern of change across the set of seven questions may be helpful in understanding why youth marijuana use decreased from 1997 to 1999. The direction of the change for six of the seven questions was consistent with the decrease in youth marijuana use. Compared with youths in 1997, youths in 1999 were more likely to have talked with parents/other adults about the dangers of substance use, reported a stronger negative reaction from parents if they were to smoke regularly, had fewer friends who used marijuana or drank alcohol heavily, participated less in gang or group fights, and had higher academic performance. Youths in 1999 also reported attending religious services less often than youths in 1997, which is not consistent with the drop in marijuana use between those 2 years.

E.4 Comparison of the Associations with Marijuana Use in 1997 and 1999

Table E.3 presents the associations with past year marijuana use of the questions that were similar but not identical between the 1997 and 1999 surveys, as well as a test of whether the differences in associations between the 2 years were statistically significant (values of p <.05). Significant differences in the associations between the 2 years were found for parental reaction to youth smoking and taking part in a gang fight in the past year. In both cases, the associations were weaker in 1999 compared with 1997. As was the case with the differences in the distributions of these items discussed above, the differences in the wording of these questions prevents a meaningful interpretation of the differences in the strengths of these associations.

For example, one explanation for the lower association seen in 1999 between past year marijuana use and parental disapproval of youth smoking compared with the association seen in 1997 between past year marijuana use and parents being upset about youth smoking is that parental reaction to youth smoking was not as predictive of youth marijuana use in 1999 as it was in 1997. A second explanation is that youths' perceptions that parents would disapprove of their smoking were not as good of a predictor of their marijuana use as were youths' perceptions that their parents would be upset about their smoking. However, the pattern of smaller associations between risk and protective factors and past year marijuana use in 1999 than in 1997 was consistent with the general trend discussed in Chapter 5.

E.5 Comparison of the Predictiveness of the Items Included in Both 1997 and 1999

In Chapter 5, prediction models were presented for both the 1997 and 1999 NHSDAs that included a set of demographic variables (age, gender, and race/ethnicity) as well as the set of 11 questions that were directly comparable between the 2 years of the survey. Those results indicated that the 1997 model accounted for a higher percentage of the variation in past year marijuana use (53 percent) than the 1999 model (47 percent).

Table E.4 presents these same prediction models after the addition of the set of seven questions that were similar but not identical between the 2 years of the survey. In this table, the set of demographic variables is included first, followed by the set of 11 questions that were directly comparable between 1997 and 1999, and finally the set of questions that were similar but not identical between the 2 years of the survey. The addition of the set of similar but not identical items increased the amount of variance accounted for by both the 1997 model (R² = 0.35; R_N² = 0.59) and the 1999 model (R² = 0.31; R_N² = 0.56). These results indicate that the 1997 model still accounts for more overall variation in youth past year marijuana use compared with the 1999 model, but that the "gap" between the two models was reduced with the addition of the variables that were similar but not identical between the 2 years of the survey.

Table E.1 Comparison of Question Wording and Response Options of Risk and Protective Factor Questions Measured Using Similar But Not Identical Questions Between the 1997 and 1999 NHSDAs

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1997 and 1999.

Table E.2 Comparison of Distributions of Risk and Protective Factor Variables Measured Using Similar But Not Identical Questions Between the 1997 and 1999 NHSDAs

Note: The 1999 NHSDA data were derived from the 1999 PAPI, with weights adjusted for field interviewer experience.
Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1997 and 1999.

Table E.3 Comparison of Unadjusted Associations with Past Year Marijuana Use of Risk and Protective Factors Measured with Similar But Not Identical Questions in the 1997 and 1999 NHSDAs

Variable	1997 NHSDA				1999 NHSDA				Test of Difference Between 1997 and 1999
		Odds Ratio	95% CI	p value		Odds Ratio	95% CI	p value	t-test value1	df	p value
Family Domain
Communication about Substance Use in Past Year	0.14	1.16	(0.94, 1.42)	.1644	0.17	1.18	(0.82, 1.70)	.3648	0.11	1	.9127
1997: Spoken with a parent/other adult about the dangers of alcohol or drug use
1999: Spoken with at least one parent about the dangers of tobacco, alcohol, or drug use
Parents' Reaction If Youth Smoked 1 or More Packs of Cigarettes per Day	1.26	3.51	(3.09, 3.98)	<.0001	0.68	1.97	(1.51, 2.57)	<.0001	-3.90	1	.0001
Very upset ('97) / Strongly disapprove ('99)
Somewhat upset ('97) / Somewhat      disapprove ('99)
Not at all upset ('97) / Neither approve nor      disapprove ('99)
Peer/Individual
Friends' Marijuana Use	2.13	8.38	(7.32, 9.59)	<.0001	2.14	8.51	(6.41, 11.28)	<.0001	-0.10	1	.9237
1997: How many of your close friends have tried marijuana once or twice?
1999: How many of your friends would you say use marijuana or hashish?
None of them
A few of them
Most of them ('97) / Most or all of them      ('99)
Friends' Alcohol Use	1.16	3.18	(2.76, 3.65)	<.0001	1.20	3.33	(2.63, 4.21)	<.0001	0.34	1	.7374
1997: How many of your close friends had 5 or more drinks once or twice a week?
1999: How many of your friends would you say get drunk at least once a week?
None of them
A few of them
Most of them ('97) / Most or all of them      ('99)
Took Part in Gang Fight in Past Year	1.11	3.02	(2.54, 3.59)	<.0001	0.74	2.10	(1.54, 2.87)	<.0001	-2.01	1	.0452
1997: How many times gotten into a gang fight?
1999: How many times taken part in a fight where a group of your friends fought against another group?
0 times
1 or 2 times
3 or more times
Number of Times Attended Religious Services in Past Year	-1.02	0.36	(0.30, 0.44)	<.0001	-0.86	0.42	(0.29, 0.61)	<.0001	0.71	1	.4770
Never, once or twice, once or twice a month      ('97) / 0 to 24 times ('99)
Every week, several times a week ('97) / 25      to 52 times, 52+ times ('99)
School Domain
Academic Performance
Grades in last completed semester	1.61	5.00	(3.41, 7.34)	<.0001	1.28	3.60	(1.94, 6.70)	.0001	-0.89	1	.3730
Mostly A's and B's; mostly B's and C's;      mostly C's and D's (<97) / A+, A, A-; B+, B,      B-; C+, C, C- (<99)
D or below (<97) / D or below (<99)

Note: The 1999 NHSDA data were derived from the 1999 PAPI, with weights adjusted for field interviewer experience.
¹ Significance tests indicate whether the interaction terms (Factor x Year) are significantly different from zero.

Table E.4 Comparison of Adjusted Associations with Past Year Marijuana Use of Risk and Protective Factors and Demographics Measured Using Identical or Similar Questions in the 1997 and 1999 NHSDAs

Identical / Similar But Not Identical Items	1997 NHSDA				1999 NHSDA
		Odds Ratio	95% CI	p value		Odds Ratio	95% CI	p value
Demographics: Identical
Age (15 to 17 vs. 12 to 14)	0.70	2.01	(1.49, 2.71)	<.0001	0.91	2.48	(1.17, 5.24)	.0178
Gender (males vs. females)	0.26	1.30	(0.96, 1.76)	.0889	0.07	1.07	(0.64, 1.79)	.7897
Race/ethnicity
Black vs. white	-0.26	0.77	(0.53, 1.10)	.1517	-0.02	0.98	(0.50, 1.92)	.9519
Hispanic vs. white	-0.07	0.93	(0.61, 1.43)	.7348	0.04	1.04	(0.56, 1.94)	.8988
Other vs. white	0.64	1.90	(1.06, 3.40)	.0325	0.53	1.70	(0.70, 4.12)	.2382
Community Domain: Identical
Availability of Marijuana	0.47	1.60	(1.38, 1.86)	<.0001	0.51	1.67	(1.31, 2.13)	<.0001
Approached by Drug Seller in Past 30 Days (yes vs. no)	1.08	2.95	(2.11, 4.13)	<.0001	0.47	1.60	(0.91, 2.81)	.1047
Family Domain: Identical
Communication about Substance Use in Past Year	-0.29	0.75	(0.53, 1.06)	.1017	-0.47	0.63	(0.34, 1.15)	.1324
Parents as Source of Social Support (yes vs. no)	-0.46	0.63	(0.47, 0.85)	.0024	-0.05	0.95	(0.54, 1.67)	.8533
Peer/Individual Domain: Identical
Perceived Risk of Marijuana Use
Low risk of using marijuana once a month	0.42	1.53	(1.26, 1.86)	<.0001	0.20	1.22	(0.82, 1.82)	.3230
Low risk of using marijuana once or twice a week	0.53	1.69	(1.37, 2.09)	<.0001	0.60	1.83	(1.16, 2.88)	.0101
Risk-Taking Proclivity
How often do you get a kick out of doing things that are a little dangerous?	0.28	1.33	(1.14, 1.54)	.0003	0.24	1.27	(0.90, 1.80)	.1715
How often do you test yourself by doing something a little risky?	-0.15	0.86	(0.72, 1.03)	.1054	0.15	1.16	(0.82, 1.63)	.3939
How often do you wear a seatbelt when you ride in the front passenger seat of a car?	-0.14	0.87	(0.75, 1.00)	.0474	0.21	1.23	(0.94, 1.61)	.1281
Religiosity
My religious beliefs are a very important part of my life	0.21	1.23	(0.90, 1.69)	.1990	-0.35	0.70	(0.47, 1.06)	.0906
My religious beliefs influence how I make decisions in my life	-0.10	0.90	(0.66, 1.24)	.5333	-0.13	0.88	(0.60, 1.28)	.4922
It is important that my friends share my religious beliefs	-0.02	0.98	(0.80, 1.22)	.8781	0.03	1.04	(0.72, 1.49)	.8524
Family Domain: Similar But Not Identical
Parents' Reaction If Youth Smoked 1 or More Packs of Cigarettes per Day?	0.48	1.62	(1.27, 2.07)	.0002	0.50	1.66	(1.10, 2.48)	.0151
Very upset ('97) / Strongly disapprove ('99)
Somewhat upset ('97) / Somewhat disapprove ('99)
Not at all upset ('97) / Neither approve nor disapprove ('99)
Peer/Individual: Similar But Not Identical
Friends' Marijuana Use	1.20	3.32	(2.65, 4.17)	<.0001	1.61	5.00	(3.00, 8.34)	<.0001
1997: How many of your close friends have tried marijuana once or twice?
1999: How many of your friends would you say use marijuana or hashish?
None of them
A few of them
Most of them ('97) / Most or all of them ('99)
Friends' Alcohol Use	-0.08	0.92	(0.76, 1.12)	.4064	-0.41	0.66	(0.42, 1.05)	.0769
1997: How many of your close friends had 5 or more drinks once or twice a week?
1999: How many of your friends would you say get drunk at least once a week?
None of them
A few of them
Most of them ('97) / Most or all of them ('99)
Took Part in Gang Fight in Past Year	0.24	1.27	(0.83, 1.96)	.2721	0.20	1.22	(0.78, 1.93)	.3784
1997: How many times gotten into a gang fight?
1999: How many times taken part in a fight where a group of your friends fought against another group?
0 times
1 or 2 times
3 or more times
Number of Times Attended Religious Services in Past Year	-0.61	0.54	(0.40, 0.74)	.0002	-0.21	0.81	(0.45, 1.47)	.4902
Never, once or twice, once or twice a month ('97) / 0 to 24 times ('99)
Every week, several times a week ('97) / 25 to 52 times, 52+ times ('99)
School Domain: Similar But Not Identical
Academic Performance	1.06	2.88	(1.58, 5.25)	.0007	0.16	1.17	(0.32, 4.25)	.8120
Mostly A's and B's; mostly B's and C's; mostly C's and D's (<97) / A+,      A, A-; B+, B, B-; C+, C, C- (<99)
D or below (<97) / D or below (<99)
Sample size	6,066				2,417
R2 (see footnote 1)	0.35				0.31
RN2 (see footnote 2)	0.59				0.56

Note: The 1999 NHSDA data were derived from 1999 paper-and-pencil interviewing (PAPI), with weights adjusted for field interviewer experience.

¹ Cox and Snell (1989) R² is a measure of the fit of the model, defined as where L(O) is the likelihood of the intercept-only model, is the likelihood of the full model, and n is the sample size.
² Recognizing that the Cox and Snell R² reaches a maximum for models that depend on the value of the estimated percentage, Nagelkerke (1991) proposed dividing the Cox and Snell measure by the maximum. In this sense, R_N² measures the absolute percentage of variation explained by the model.

This page was last updated on July 14, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.