McGraw-Hill OnlineMcGraw-Hill Higher EducationLearning Center
Student Center | Instructor Center | Information Center | Home
Newsletter
Career Opportunities
Chapter Outline
Discussion Questions
Multiple Choice Quiz
Glossary
Flashcards
Test Developer Profiles
Web Links
Feedback
Help Center


Psychological Testing and Assessment Book Cover
Psychological Testing and Assessment: An Introduction To Tests and Measurement, 5/e
Ronald Jay Cohen
Mark Swerdlik

Reliability

Test Developer Profiles

Thomas E. Brown, Ph.D.

Test Developed:
Brown Attention Deficit Disorder Scales

Thomas E. Brown, Ph.D., is Assistant Clinical Professor of Psychiatry at the Yale University School of Medicine and Associate Director of the Yale Clinic for Attention & Related Disorders. He was born in Minneapolis, Minnesota, in 1942 and grew up in Chicago, Illinois. Dr. Brown did his undergraduate studies at Knox College in Galesburg, Illinois, after which he completed a Ph.D. in Clinical Psychology at Yale University.

The Brown Attention Deficit Disorder Scales, published by The Psychological Corporation in San Antonio, Texas, were developed out of conversations with a series of high school and college students who complained of chronic problems in maintaining their concentration, in remembering what they had read, and in getting themselves activated and organized to do their academic work. Many of these individuals were very bright, well- motivated students, but their grades were low due to insufficient studying, poor notetaking, and many late, incomplete, or undone assignments.

It took several years of talking with such students for me to recognize that many of them were suffering from attention deficit disorders even though they were not hyperactive. At that time most clinicians thought of attention deficit disorders primarily as a syndrome of behavioral problems in young children; and most rating scales for ADD tapped problems of hyperactivity and impulsivity with few items querying inattention and short-term memory problems. As I recognized similarities among these nonhyperactive students with attention deficit disorder (ADD), I began gradually to collect a series of symptom descriptors that were recurrently reported. From the resulting item pool I developed a self-report rating scale for cognitive symptoms of ADD, which I tested on groups of adolescents and adults--some already diagnosed with ADDs and others in nonclinical settings. I tested this 40-item scale on 191 adolescents and 142 adults, all of whom met DSM-III diagnostic criteria for ADD with and without hyperactivity, and on 190 nonclinical controls matched for age and SES.

Scale items derived from self-report of ADD patients were not fully congruent with the DSM-III or DSM-III-R symptom lists, so it became important to determine how well the various items fit together. I did this by using item-total correlations to determine how well each item fit with total score; I also utilized Cronbach's Coefficient Alpha as a measure of internal consistency for the scale. These measures showed that items derived from ADD patient self-descriptions fit into a coherent whole with items already established as canonical symptoms of attention deficit disorder.

Our next step was to administer the instrument to a nonclinical sample matched with the clinical sample for age and SES. This was done to ascertain whether this instrument could adequately differentiate between persons with ADD and others without ADD: Could it tell ADD patients from normal controls?

My assumption was that most symptoms of ADD are experienced by virtually everyone from time to time. Presumably persons who have ADD are those who experience significant and persistent impairment from ADD symptoms. Thus, if this new rating scale was to be useful, it would need to be able to differentiate between persons who report significant impairment from these ADD symptoms and others who do not report such impairment. I needed to compare the scores of persons who had already been diagnosed as having met diagnostic criteria for ADD and a community sample where one would expect, on the basis of existing epidemiological estimates, to find only about 5% of the sample to have ADD.

After administering the instrument to samples of normal controls, I compared the overall scores of the clinical and nonclinical samples using a t test to assess differences between the means and Cohen's d to measure the effect size of these differences. These analyses showed highly significant differences between the means of the two groups and very adequate effect sizes for the differences.

To use this rating scale as both a screening instrument and a tool to assist diagnostic assessment, it was necessary to provide a score to be used as the clinical cut-point for differentiating persons who probably would meet diagnostic criteria for ADD and those who probably would not. To do this I utilized the ROC, receiver operating characteristics, to determine the percent of false positives and false negatives associated with possible cut-scores. Given the 40 items of the scale and a scoring system of 0 to 3, an individual's score on this scale could range from 0 to 120. Our ROC analysis indicated that setting the cut-score at 50/120 in this sample would yield 22% false positives and 10% false negatives.

In other words, if we took scores of the 191 adolescents diagnosed with ADD and of the 190 nonclinical adolescents and identified all those scoring 50 or higher on the scale as probably having ADD, we would have missed only 10% of those who had previously been identified as having ADD, and we would have mistakenly designated 22% of the nonclinical sample as probably having ADD.

For purposes of screening, that is, doing a preliminary check to identify persons whose reported symptoms might warrant a more comprehensive assessment for possible ADD, the cut-score of 50 seemed to offer an optimal balance between sensitivity and specificity. It minimized the number of false negatives and held a reasonable percentage of false positives to be cleared out later by other measures. In the manual we report false positive/false negative percentages associated with six other possible cut-scores. This allows clinicians or researchers to select another cut-score that might be more appropriate for their use of the scale.

In addition to screening and comprehensive assessment for possible ADDs, the Brown scales are also intended for monitoring effects of treatment on an individual's ADD symptoms. In order to use repeated administrations of the Brown scales for this purpose it was necessary to control for two kinds of variability: variability caused by regression to the mean and variability caused by the imperfect reliability of the instrument. This control was obtained in a novel way, by creating confidence intervals around the estimated true score obtained prior to treatment.

These confidence intervals, based on the standard error of prediction, set for each obtained score a range against which scores obtained in subsequent assessment of the individual can be tested to see if they reflect sufficient change from the originally obtained score to claim a statistically significant difference. Use of these confidence intervals helps to reduce the likelihood of error in assessing whether medication or other treatment interventions for ADD are having an effect on reported symptoms.

More detailed descriptions of the psychometrics and underlying concepts of the Brown ADD Scales are included in the published manual for this instrument, which provides evaluators a simple, systematic, and effective way to elicit self-report data on ADD symptoms from adolescents (12--18 years) and adults (18 and up). A new adaptation of the Brown ADD Scales to be utilized with children aged 6 to 12 years is currently being standardized and will soon be published by The Psychological Corporation.