Consistencies in self-reported drug use among students in Ontario (Canada)


Sampling design and procedure
Questionnaire and its administration


Pages: 55 to 65
Creation Date: 1985/01/01

Consistencies in self-reported drug use among students in Ontario (Canada)

M. S. GOODSTADT , * Head,
G. C, CHAN, * Senior Research Assistant
M. A. SHEPPARD *and Research Associate
J.C. CLEVE, * Research Assistant
Education Research Section, Addiction Research Foundation, Toronto, Canada


A survey of drug use, carried out in 1981 by means of a self-reported anonymous questionnaire administered to a stratified probability sample of 4,306 school students in grades 5, 7, 9, 11 and 13 in Ontario, Canada, found consistent patterns in the use of drugs, including infrequently used illegal drugs such as heroin. Consistency of drug use was more apparent in the school grades with greater prevalence rates of drug use. Drug use was unevenly distributed and was localized within a relatively small proportion of all schools and geographical areas covered. Among the l7 categories of substances studied, the least frequently used substances were most localized. The study shows a general absence of misrepresentation of drug use by students, which corroborates findings reported by other researchers who have indicated that the problems of reliability and validity of self reported drug use appear to be more related to difficulties in recalling and self-defining drug use than to its deliberate misrepresentation. The evidence presented in the article suggests that data on self-reported student d rug use, as found in this and similar surveys, can be accepted with reasonable confidence.


This article is concerned with consistencies in self-reported drug use among student populations. Series of surveys of drug use among student populations have been conducted for many years [ 1] - [ 3] . A review of the evidence [ 4] - [ 6] indicates that there are relatively little data concerning the reliability and validity of information obtained from these surveys, that acceptable but not uniform levels of test-retest reliability have been reported [ 7] , [ 8] , and that the reliability and validity of reported drug use appear to be related more to problems of recall and self-definition than to deliberate misrepresentation of prior drug use. Evidence also indicates that a greater proportion of inconsistencies in reported drug use is likely to occur when the use of a particular drug is uncommon, and in reported non-use where the behaviour is common [ 9] .

* The views expressed in this article are those of the authors and do not necessarily reflect the views of the Addiction Research Foundation.

The reliability and validity of reported drug use continues to be a focus of concern for both clinicians and researchers. Considerable attention has been given to the confidence one can place in reports of alcohol use by individuals in treatment [ 10] - [ l3] , and to discrepancies in self-reported alcohol use as compared to reports by "significant others" [ 14 ] - [ 16] . Problems in obtaining valid estimates of alcohol Consumption, especially as derived from self-reports, have been examined at both the societal [ 17] , [ 18] and individual levels [ 19] , [ 20] . Researchers have made efforts to develop reliable and valid instruments for assessing drug use and abuse, but more attention has been given to alcohol use than to the use of other substances [ 21] - [ 27] ; broader drug use and abuse scales have also been developed [ 28] , [ 29] . Factors associated with invalidity of instruments, as well as methods to overcome the sources of bias or error in measuring drug use and abuse have been examined in a number of studies [ 30] - [ 39] .

This article describes the consistencies of drug use data obtained from a 1981 survey of school students ; this was one in a series of surveys conducted every two years since l968. Particular attention is given to heroin use because of the seriousness of such use and the media response to the problem. Data on the consistencies of reported drug use in this survey are presented with respect to the schools and classes included in the sample for this survey, as well as the geographical areas which the survey covered. Data are also analysed and presented with respect to the reported use of each of 17 categories of substances studied in this survey, as well as of these substances considered collectively.

Sampling design and procedure

For the 1981 survey a stratified single-stage probability sample was selected. The data were weighted to take into account the sample design and non-response by selected schools and students. The sample design was estimated to cover 93 per cent of the population of Ontario students enrolled in grades 5, 7, 9, l1 and 13, the age of the students was l0, 12, 14, 16 and 18 years, respectively.

The stratification of the sample included four geographical areas (Metro Toronto, West, East and North Ontario) and five grades (5, 7, 9, 11 and 13), resulting in 20 strata. The 1981 projected student enrolment for each stratum was calculated on the basis of enrolment figures from the two prior academic years : 1978/79 and 1979/80. A probability sample of students was independently selected from each of the 20 strata. The sampling unit was a homeroom class, either as it existed in a given school or as determined on the basis of an average class size of 30 students. In anticipation of refusals to participate in the study, two additional homeroom clusters were randomly selected from each stratum.

Of 34 school boards that were requested to take part in the study, seven refused entirely to participate, while a few refused to allow participation by their grade 5 students ; four substitute boards were employed so that 3l boards took part in the study. Once board approval was obtained, the schools were contacted and arrangements were made to administer the survey. The 3l cooperating boards comprised of 259 selected schools, which included 280 homeroom classes. For various reasons, 54 schools refused to participate ; a further 23 schools were eliminated from analyses because principals had intervened in the selection process. Therefore, a total of 182 schools, consisting of 198 homeroom classes, were therefore included in the sample for analysis. Weights were applied to correct the data for the cluster characteristics of the sample, and for non-response at the school and student levels. The unweighted sample consisted of a total of 4,306 students ; of these, 1,036 students were in grade 5,782 in grade 7, 1,004 in grade 9,894 in grade 11 , and 590 in grade 13. The average response rate at the student level was 85.2 per cent, ranging from lows of 8l.0 per cent and 79.6 per cent for grade 11 students and the Metro-Toronto area to highs of 90.4 per cent and 91.l per cent for grade 7 students and the Northern Ontario area.

Questionnaire and its administration

The data-gathering instrument was a self-administered anonymous questionnaire, which was available in English and French. The questionnaire included questions on the frequency of drug use in the preceding l2 months, to which students responded by means of multiple choice alternatives. "Non-use" and not knowing the drug in question were also part of the multiple choice alternatives. Except for the use of tobacco and alcohol, "drug use" in this report refers to any use of the drug within the 12 months prior to the survey. Tobacco use refers to any use other than "I tried one cigarette in the last 12 months". Alcohol use refers to any use other than "I have had a sip of alcohol to see what it is like".

The students were administered questionnaires in class groups, in 30 to 40 minute sessions, during the months of February and March, 198l . Participation was voluntary. Field staff provided a short introduction to the study for students prior to its administration. Students recorded their responses directly on the questionnaire.


Consistencies in reported drug use by geographical area and substance

Patterns of drug use prevalence were examined for each school and for each of the l7 categories of substances included in this study. Average prevalence rates were calculated for those schools in which multiple classes were sampled within a given grade, which allowed for comparisons to be made between schools. The data obtained from this first level of analysis support the following conclusions :

  1. Drug use reported in this survey is not distributed randomly ;

  2. The prevalence of drug use is unevenly distributed, being localized within a relatively small proportion of all schools and geographical areas covered by the survey ;

  3. Higher prevalence rates of drug use are more common within a small proportion of those schools which report any drug use ;

  4. The use of substances is least localized in the case of more socially accepted and more commonly used substances, such as alcohol, and most localized in the case of least socially accepted substances, such as heroin.

A second level of analysis examined the combined pattern of drug use for l7 categories of substances within individual schools. It was found that certain schools consistently showed higher rates of the use of certain categories of substances. The consistency with which a given school is among the subset of schools accounting for 50 per cent of drug users (with respect to the use of all 17 drug categories) was analysed for each school grade. For example, school number l38 (for grade 5) was one of the "upper 50 per cent" schools with respect to 12 of the l7 categories of substances ; this was one instance of high drug concentration. A given school was classified in the subset of schools with high drug concentration if drug use in the school was highly localized for 8 or more of the l7 drug categories of substances. Patterns of localization suggest that :

  1. A greater localization of drug use exists in grades 9 and 11 than in grades 5, 7, or 13;

  2. The concentration of drug use does not only occur within a few schools, but it also occurs within certain geographical areas covered by the survey. Thus, highly localized drug use is more common in the eastern region in the lower grades (5 and 7) but more concentrated in the western region in grades 9 and 11. These findings support the conclusion that not only is the prevalence of use of any particular substance unevenly distributed throughout the sample and across the Province, but that this consistency in drug use is also found when drug categories are considered together.

A third level of analysis of the localization of drug use across the 17 categories of substances is obtained, for each school grade separately, by computing :

  1. The percentage of drug users in each class, including a correction for differences in the size of classes;

  2. The rank-order of these percentages, within each grade and category of substances;

  3. The median of the 17 computed ranks (i. e. across the 17 categories of substances) within each grade;

  4. The rank-order of these median ranks for the schools within each grade.

This analysis, which permits the identification of schools, and the corresponding geographical areas, with the highest concentration of drug use for all 17 categories of substances, shows that drug use was localized, for each grade, within geographical areas and within particular boards (sub-areas).

Heroin use followed the same geographical distribution as shown for the combined use of substances. There were only 48 cases of reported heroin use. The distribution of heroin users was localized in the same boards as users of other substances; 56 per cent of the cases of heroin use occurred in the western region, especially in grades 9 and 11, and 31 per cent in the eastern region.

These findings suggest that reported heroin use is not randomly distributed throughout the Province; it follows the geographical distribution found for the combined use of other substances.

Consistencies in reported drug use within the survey

Questions within the survey provide the basis for an examination of the consistency of reported drug use. The data considered so far were obtained from a series of 17 questions inquiring, for each of the 17 substances included in this study, about the frequency of drug use within the previous 12-months. A further question was asked (except for grade 5 students) about the order in which the 17 categories of substances had been first used (if ever) in the respondent's lifetime. Responses to this additional question show whether a given substance had ever been used in a respondent's lifetime; by comparisons with responses to the questions about drug use in the previous 12 months, the following "truth" table was established:


Drug use in the month prior to the survey

Drug use at any point in lifetime



1 True
2 False positive
3 Unknown 4 True

The two cells of significance for the present discussion are Nos. (2) and (3). Cell No. (2) helps to show the respondents who claim to have used a drug in the previous 12 months but never to have used the drug at any point in their lifetime; these cases are false positives. Cell No. (3) helps to indicate false negatives, that is, those who falsely reported no drug use in the previous 12 months; this cell, however, may include a considerable number of former drug users [ 40] who have not used drugs in the previous 12 months. Without longitudinal data, it is difficult to establish the definite existence of false negatives in cell No. (3) [ 9] .

The table below shows the rates of false-positive responses for each of the 17 categories of substances for grades 7, 9, 11 and 13. The table also includes both uncorrected and corrected (for false positive responses) percentage prevalence rates of drug use. Such corrections have been made on the assumption that the error occurred in reporting drug use in the past 12 months rather than in erroneously reporting that the drug had never been used. Corrected prevalence = [uncorrected prevalence - (false-positive rate x uncorrected prevalence)]. The Pearson correlation coefficient between prevalence and false-positive rates was calculated for the data presented in the table below; it ranged from 0.26 to 0.36.

Prevalence, false positive rates


Grade 7

Grade 9


Uncorrected prevalence (Percentage)

False positive rate

Corrected prevalence (Percentage)

Uncorrected prevalence (Percentage)

False positive rate

Corrected prevalence (Percentage)

14.9 0.037 14.3 32.6 0.022 31.9
53.1 0.125 46.5 71.5 0.067 66.7
5.2 0.094 4.7 25.1 0.043 24.0
Glue b
4.7 0.214 3.7 4.0 0.102 3.6
Solvents c
7.0 0.194 5.6 3.7 0.000 3.7
Medical use of
7.6 0.638 2.8 11.3 0.515 5.5
Stimulants d
4.4 0.531 2.1 5.5 0.453 3.0
4.2 0.231 3.2 6.5 0.262 4.8
Non-medical use of
1.5 0.504 0.7 6.4 0.341 4.2
Stimulants d
3.4 0.341 2.2 15.5 0.212 12.2
2.0 0.572 0.9 6.9 0.239 5.3
1.0 0.098 0.9 6.8 0.140 5.8
Lysergic acid diethylamide
2.0 0.270 1.5 9.5 0.065 8.9
Phencyclidine (PCP)
1.0 0.103 0.9 3.0 0.175 2.5
Other hallucinogens
0.9 0.392 0.5 6.4 0.226 5.0
2.8 0.167 2.3 4.6 0.101 4.1
1.5 0.291 1.1 2.4 0.273 1.7
3.4 0.270 2.2 6.5 0.175 5.0

bFor example, aeroplane glue and contact cement used for sniffing.

cFor example, nail polish remover, paint thinner and petrol used for sniffing.

dFor example, diet pills.

The table below shows that:

  1. False positives range from 0.00 (or 0.0 per cent) to a maximum of 0.63 (or 63 per cent);

  2. The rate of false positives tends to decline with age, from a median of 0.27 (or 27.0 per cent) for grade 7 to 0.088 (or 8.8 per cent) for grade 13;

  3. The rate of false positives varies across drugs. Examination of the table below suggests that the highest rates of false-positive responses occur with respect to reporting the use of barbiturates, stimulants and tranquillizers, especially medical drugs. The lowest rates of false positives occur, as might have been expected [ 9] , in relation to the most frequently used drugs such as tobacco, cannabis and alcohol.

and corrected prevalence aof drug use


Grade 11

Grade 13


Uncorrected prevalence (Percentage)

False positive rate

Corrected prevalence (Percentage)

Uncorrected prevalence (Percentage)

False positive rate

Corrected prevalence (Percentage)

Uncorrected prevalence (Percentage)

False positive rate

Corrected prevalence (Percentage)

44.6 0.002 44.5 30.0 0.003 29.9 29.1 0.015 28.7
89.0 0.0011 88.0 90.6 0.013 89.4 71.7 0.056 67.7
42.1 0.031 40.8 36.5 0.012 36.1 23.7 0.035 22.9
1.6 0.000 1.6 0.4 0.000 0.4 3.2 0.143 2.7
2.1 0.254 1.6 0.9 0.265 0.7 4.1 0.146 3.5
14.9 0.442 8.3 11.9 0.416 6.9 11.0 0.504 5.5
6.5 0.228 5.0 4.5 0.405 2.7 5.2 0.399 3.1
9.3 0.170 7.7 7.3 0.120 6.4 6.5 0.202 5.2
12.0 0.296 8.4 6.4 0.198 5.1 6.0 0.313 4.1
29.0 0.120 25.5 23.0 0.079 21.2 15.4 0,154 13.0
6.7 0.141 5.8 5.4 0.125 4.7 5.0 0.235 3.8
5.4 0.123 4.7 2.0 0.169 1.7 3.9 0.133 3.4
16.5 0.034 15.9 9.3 0.019 8.4 8.6 0.057 8.1
2.6 0.000 2.6 1.1 0.000 1.1 2.0 0.094 1.8
11.5 0.083 10.5 8.4 0.088 7.7 6.0 0.140 5.2
5.0 0.000 5.0 5.0 0.096 4.5 4.1 0.085 3.8
1.6 0.331 1.1 0.3 0.000 0.3 1.6 0,284 1.1
9.3 0.120 7.7 6.4 0.088 5.1 6.0 0.145 4.1

aCorrected prevalence = [uncorrected prevalence - (false-positive rate x uncorrected prevalence)].

With regard to heroin use, false-positive rates are ranked among the highest and are fairly consistent across grades other than 13. Prevalence rates of the use of heroin and other substances that are very low are little affected in absolute terms by large false-positive rates. For example, the prevalence rate for heroin in grade 1l is 1.6 per cent and it is reduced to a corrected prevalence rate of 1.1 per cent, though the false-positive rate for heroin is very high 0.331 (or 33.1 per cent).


Results of the present research indicated that self-reported use of drugs even of very infrequently used illegal drugs such as heroin, exhibited consistent patterns of use, which suggested that there was a general absence of misrepresentation of drug use by students of any grade participating in the survey. These data are in favour of accepting such survey results as being within tolerable bounds of accuracy.

Consistency of drug use occurred in several significant domains ; the use of each of the 17 categories of substances was highly concentrated within a relatively small proportion of schools ; and consistency was also shown when the combined use of substances was considered both within schools and geographical areas.

Patterns of consistency of drug use were more apparent in those grades with greater overall prevalence rates of drug use, such as in grades 9 and 11 , which also showed marked geographical consistency. The lower grades and grade l3 showed considerable consistency within schools and some consistency within geographical areas.

The single-time period of the present study does not permit an examination of either how these patterns change over time or the relationship between patterns of use in earlier grades and later grades [ 2] .

In summary, the results of this survey lend further support for reliability and validity of data on self-reported student drug use. This conclusion should not be taken to mean that no sources of error existed or that no student dissembled, but the obtained data show that the overall findings of this and similar surveys can be accepted with reasonable confidence.


The authors are grateful to Dr. P. Peskun and Dr. M. Lanphier of York University, Toronto, for developing the sample for this study and for their advice on several stages in the design and analysis of data. The authors also wish to thank the Survey Research Centre, Behavioural Research Institute, York University, Toronto, for taking the responsibility for the administration of the study, data collection, data weighting and initial data processing.



l. R. G. Smart and others, Preliminary Report of Alcohol and Other Drug Use Among Ontario Students in 1983, and Trends Since 1977 (Toronto, Alcoholism and Drug Addiction Research Foundation , l983).


M . S. Goodstadt, G. C. Chan and M. A. Sheppard, "Developmental and generational trends in alcohol, cannabis and tobacco use - A ten year cohort analysis", Drug and Alcohol Dependence, vol. 10, 1982, pp. 303 - 320.


L. D. Johnston, J. G. Bachman and P. M. O'Malley, Student Drug Use. Attitudes, and Beliefs. National Trends 1975-1982 (Rockville, Maryland, National Institute on Drug Abuse, l982).


P. C. Whitehead and R. G. Smart, "Validity and reliability of self-reported d rug use", Canadian Journal of Criminology and Corrections, vol. 14, 1972, pp. 1 - 7.


R. G. Smart, " Recent studies of the validity and reliability of self-reported drug use l970 - l974", Canadian Journal of Criminology and Corrections, vol. 17, 1975 , pp. 326 - 333.


R. G. Smart and G. K. Jarvis, "Do self-report studies of d rug use really give dependable results?", Canadian Journal of Criminology and Corrections, vol. 23, No. 1 (1981), pp. 83 - 92.


J. T. Gossett, J. M. Lewis and V. A. Phillips, "Extent and prevalence of illicit drug use as reported by 56,745 students", Journal of the American Medical Association, vol. 216, 1971, pp. 1464 - 1470.


R. G. Smart and N. L. Blair, "Test-retest reliability and validity information for a high school drug use questionnaire", Drug and Alcohol Dependence, vol. 3 , l978, pp. 265 - 271 .


E. Single, D. Kandel and B. D. Johnson , "The reliability and validity of d rug use responses in a large scale longitudinal survey", Journal of Drug Issues, vol. 5, 1975 , pp. 426 - 443.


E. J. Freedberg and W. E. Johnston, "Validity and reliability of alcoholics' self reports of use of alcohol submitted before and after treatment", Psychological Reports, vol. 46, 1980, pp. 999 - 1005.


H. Orrego and others, "Reliability of assessment of alcohol intake based on personal interviews in a liver clinic", The Lancet, 1979, pp. 1354 - 1356.


H. A. Skinner, "Assessing alcohol use by patients in treatment ", Research Advances on Alcohol and Drug Problems, vol. 8, R. G. Smart and others, eds. (New York : plenum, in press).


L. C. Sobell and M. B. Sobell, "Validity of self-reports in three populations of alcoholics", Journal of Consulting and Clinical Psychology, vol. 46, No. 5 (1978), pp. 90l - 907.


W. N. Friedrich and S. O. Lofts gard, "Comparisonof two alcoholism scales with alcoholics and their wives", Journal of Clinical Psychology, vol. 34, No. 3 (1978), pp. 784 - 786.


B. S. McCrady, T. J. Paolino and R. Longabaugh, "Correspondence between reports of problem drinkers and spouses on drinking behaviour and impairment ", Journal of Studies on Alcohol, vol. 39, No. 7 (1978), pp. 1252 - 1257.


W. R. Miller, V. L. Crawford and C. A. Taylor, "Significant others as corroborative sources for problem drinkers", Addictive Behaviors, vol.4, 1979, pp. 67 -70.


K. Pernanen, "Validity of survey data on alcohol use", Advances in Alcohol and Drug Problems, vol. 1, R. J. Gibbons and others, eds. (New York, John Wiley, 1974).


R. E. Popham and W. Schmidt, "Words and deeds: the validity of self-report data on alcohol consumption", Journal of Studies on Alcohol, vol. 42, No. 3 (1981), pp. 355 - 368.


D. C. Uchalik, "A comparison of questionnaire and self-monitored reports of alcohol intake in a nonalcoholic population", Addictive Behaviors, vol. 4, 1979, pp. 409 413.


L. Midanik, "The validity of self-reported alcohol consumption and alcohol problems: a literature review", British Journal of Addiction, vol. 77, 1982, pp. 357 -382.


D. D. Celentano and D. V. McQueen, "Reliability and validity of estimators of alcoholism prevalence", Journal of Studies on Alcohol, vol. 39, No. 5 (1978), pp. 869-878.


T. W. Dunlop, "The alcadd test: an extension of norms ", American Journal of Drug and Alcohol Abuse, vol. 5, No. 2 (1978), pp. 211 -220.


M. R. Hilton and V. G. Lokare, "The evaluation of a questionnaire measuring severity of alcohol dependence", British Journal of Psychiatry, vol. 132, 1978, pp. 42 -48.


R. Hollard and others, "Reliability of an alcohol self-report instrument", Journal of Studies on Alcohol, vol.40, No. 1 (1979), pp. 142-144.


K. A. Khavari, P. D. Farver and F. M. Douglass IV, "A scale for the indirect assessment of alcohol intake. The alcohol use predictor scale", Journal of Studies on Alcohol, vol.40, No. 7 (1979), pp. 683 -692.


J. E. Mayer and W. J. Filstead, "Empirical procedures for defining adolescent alcohol misuse", Journal of Alcohol Studies, vol. 40, No. 3 (1979), pp. 291 -300.


J. E. Mayer and W. J. Filstead, "Empirical procedures for defining adolescent alcohol misuse", Adolescence and Alcohol, J. E. Mayer and W. J. Filstead, eds. (Cambridge, Mass.: Bellinger Publishing Co. (Extended version of Mayer and Filstead, 1979).


K. Kupetz and others, "The acceptance and validity of the substance use and abuse survey (SUAS)", Journal of Drug Education, vol. 9, No. 2 (1979), pp. 163 -180.


H. A. Skinner, "The drug abuse screening test", Addictive Behaviors, vol. 7, 1982, pp. 363 -371.


E. Blair and others, "How to ask questions about drinking and sex: response effects in measuring consumer behavior", Journal of Marketing Research, vol. 14, 1977, pp. 316-321.


J. Carifo and R. Biron, "Collecting sensitive data anonymously : the CDRGP technique", Journal of Alcohol and Drug Education, vol. 23 , N o. 2 (1978), pp. 47 - 66.


K. C. Dube and others, "The question of onymity and anonymity of a questionnaire in a drug use survey", Bulletin on Narcotics (United Nations publication), vol. 33, No. 2 (1981), pp. 45 - 48.


M. S. Goodstadt, G. Cook and V. Gruson, "The validity of reported d rug use : the randomized response technique", The International Journal of the Addictions, vol. l3, No. 3 (1978), pp. 359 - 367.


M. J. Luetgert and A. H. Armstrong, "Methodological issues in drug usage surveys : anonymity, recency and frequency", The International Journal of the Addictions, vol. 8, No. 4 (1973), pp. 683 - 689.


J. H. Malvin and J. M. Moskowitz, "Anonymous versus identifiable self-reports of adolescent drug attitudes, intentions, and use", Public Opinion Ouarterly, vol. 47, No. 4 (1983), pp. 557 - 566.


M. A. Plant, J. Chick and N. Kreitman, "The effects of response rates on levels of self-reported alcohol consumption and alcohol-related problems : conclusions from a Scottish study", British Journal on Alcohol and Alcoholism, vol. 15 , No. 4 (1980), pp. 158 - 163.


R. G. Smart and C. B. Liban, "Alcohol consumption as estimated by the informant method ; a household survey and sales data", Journal of Studies on Alcohol, vol. 43 , No. 9 (1982), pp. 1020 - l027.


B. D. Smith and V. Nacev, "Drug usage as determined under conditions of anonymity and high questionnaire return rate ", The International Journal of the Addictions, vol. 13 , No. 5 (1978), pp. 725 - 736.


A. Zanes and E. Matsoukas, "Different settings, different results? A comparison of school and home responses", Public Opinion Ouarterly. vol. 43 , No. 4 (1979), pp. 550 - 557.


M. S. Goodstadt, M. A. Sheppard and G. C. Chan, "Non-use and cessation of cannabis use : neglected foci of drug education", Addictive Behaviors, vol. 9, 1984, pp. 21 - 31 .