Test Developer Profiles

Jack A. Naglieri, Ph.D.

Tests Developed:

Cognitive Assessment System

Devereux Behavior Rating Scale-School Form

Devereux Scales of Mental Disorders

General Ability Scale for Adults

Matrix Analogies Test

Naglieri Nonverbal Ability Test

From early in my educational career I have had a strong interest in how people learn and what variables interfere with knowledge acquisition. During my undergraduate studies I was fortunate to have excellent training in experimental and physiological psychology, which eventually became very important to the development and theory behind the Cognitive Assessment System (CAS; Naglieri and Das, 1997a). These areas of interest were further developed during my master's degree studies and initial certification as a school psychologist.

During my graduate studies at St. John's University, I was introduced to the work of A. R. Luria by a professor (Dr. John Carboy) who saw the importance of cognitive and cognitive and neuropsychology to the practice of school psychology. His emphasis on neuropsychology had considerable influence on me at the time, but I had no idea that many years later, I would use Luria's view of the three functional units of the brain to redefine intelligence. The most significant event in my educational training, however, came during the next phase of my formal training, when I was a Ph.D. student at the University of Georgia. Clearly, the most influential person of my educational training was my advisor, Dr. Alan S. Kaufman. In addition to sharing his vast knowledge about testing, he introduced me to the process of test development and gave me the opportunity to help develop tests that eventually became part of his own work (e.g., the Kaufman Assessment Battery for Children; Kaufman and Kaufman, 1983). I then went on to use this knowledge to create my own tests.

The knowledge I acquired at the University of Georgia, combined with my previous training and practical experiences, allowed me to create my own tests. My first projects were the Matrix Analogies Test, Expanded and Short Forms (Naglieri, 1985a, 1985b) and the Draw A Person: Quantitative Scoring System (Naglieri, 1988). All of these tests were developed at about the same time and jointly standardized. This started my career as a test developer and led to the publication of Draw A Person: Screening Procedure for Emotional Disturbance (Naglieri, McNeish, and Bardos, 1991), the Devereux Behavior Rating Scale-School Form (Naglieri, LeBuffe, and Pfeiffer, 1993), the Devereux Scales of Mental Disorders (Naglieri, LeBuffe, and Pfeiffer, 1994), the Devereux Scoring Assistant (LeBuffe, Naglieri, and McNeish, and Bardos, 1991), the Devereux Behavior Rating Scale-School Form (Naglieri, LeBuffe, and Pfeiffer, 1993), the Devereux Scales of Mental Disorders (Naglieri, LeBuffe, and Pfeiffer, 1994), the Devereux Scoring Assistant (LeBuffe, Naglieri, and Pfeiffer, 1996), the Naglieri Nonverbal Ability Test (Naglieri, 1997), and the General Ability Measure for Adults (Naglieri and Bardos, 1997). All of these tests were published during the time I worked on the Cognitive Assessment System (CAS)–"this test took the longest to develop because it is the most complex and involved development on a much larger scale. I learned that making a major intelligence test doesn't happen overnight The development of the CAS began on a cold February day in 1984 when J. P. Das and I met for the first time. Our compatibility was apparent from the beginning and, fortunately, continued because there was much work to do. Our efforts began with many discussions of what the field had–"tests built on vague theories of intelligence mainly divided by verbal and nonverbal content following from the Army Testing Program. Next we focused on what we thought the field needed–"a theory of intelligence based on recent findings in cognitive and neuropsychology and a practical test designed according to that theory. This was our grand plan, next we faced the task of experimentation with the tests we developed, the process of publishing our findings in the scientific literature (see Das, Naglieri and Kirby, 1994 for a summary), and the eventual publication of the CAS.

The process of development of the CAS was one with many ups and downs, successes and challenges, and many lessons in persistence and patience. The most significant thing I learned was that there are many roadblocks to changing professional attitudes and practice. I also learned that the best way to facilitate change is to show the value of the work through good science and clear logic and at the same time meet the needs of the profession. Because we took the posture that change was needed, we had the added requirement to provide evidence of the advantage of the PASS Theory and the CAS. We met this need by providing the results of extensive validity research, test interpretation, and intervention implications in the CAS Interpretive Handbook (Naglieri and Das, 1997b).

Pros and Cons of Being a Test Developer

There are many advantages to being a test developer, but the most important one is that I can provide tools that allow us to do our job better. Psychological tests are the tools of the trade, the better the tool the more effective we can be. Importantly, the CAS provides a tool that expands the concept of intelligence to areas not included in traditional IQ tests. This allows the practitioner to see strengths and weaknesses not detected by IQ tests and make more informed decisions about what is wrong and what can be done to improve a child's performance. Moreover, CAS is the first major shift away from "g" oriented measures of intelligence that we have used since the early 1900s. The satisfaction of knowing that a test I developed could have a positive impact on the field of psychology and ultimately help children is the greatest advantage of test development.

The disadvantage of test development is that it takes time, lots of hard work, and a knowledge of many topics including test development, psychometrics, statistics, many fields of psychology, business, marketing, promotion, and so on. It takes personal creativity and an understanding of practitioners and their needs. Test development is a task that requires a person be comfortable at the point where science, professional practice, and business intersect. In addition, a person has to be able to work for several or many (depending on the size of the project) years prior to having a finished product.

Advice to Prospective Test Developers

During the past ten years we have seen a tremendous change in our society brought about by the advances in technology. The CAS is the best example of my attempt to change the technology used by psychologists for the betterment of the field. In order for the field to change, psychologists have to be convinced that new technology is substantially better. Prospective test developers should consider their role in advancing the technology used by psychologists. To advance the field of psychology, test developers should look to introduce new methods and concepts to the field. Their products should be carefully tested and shown to do what the test developer suggests they can do, and the products' advantages should be demonstrated to practitioners–"because ultimately the value of any new psychological technology is directly related to its effectiveness in the field.

References

Das, J. P., Naglieri, J. A., and Kirby, J. R. (1994). The assessment of cognitive processes: The PASS theory of intelligence. Boston: Allyn and Bacon.

Kaufman, A. S., and Kaufman, N. L. (1983). Kaufman Assessment Battery for Children. Circle Pines, MN: American Guidance Service.

LeBuffe, P. A., Naglieri, J. A., and Pfeiffer, S. I. (1996). Devereux Scales of Mental Disorders-Scoring Assistant. San Antonio: The Psychological Corporation. Naglieri, J. A. (1985a). Matrix Analogies Test-Expanded Form. San Antonio: The Psychological Corporation.

Naglieri, J. A. (1985b). Matrix Analogies Test-Short Form. San Antonio: The Psychological Corporation.

Naglieri, J. A. (1988). Draw A Person: A Quantitative Scoring System. San Antonio: The Psychological Corporation.

Naglieri, J. A. (1996). Naglieri Nonverbal Ability Test. San Antonio: The Psychological Corporation.

Naglieri, J. A. (1997). Naglieri Nonverbal Ability Test-Individual Form. San Antonio: The Psychological Corporation.

Naglieri, J. A., and Bardos, A. N. (1997). General Ability Scale for Adults (GAMA). Minnetonka, NM: National Computer Systems.

Naglieri, J. A., and Das, J. P. (1997a). Cognitive Assessment System. Chicago: Riverside.

Naglieri, J. A., and Das, J. P. (1997b). CAS Interpretive Handbook. Chicago: Riverside.

Naglieri, J. A., McNeish, T. J., and Bardos, A. N. (1991). Draw A Person: Screening Procedure for Emotional Disturbance. Austin: ProEd.

Naglieri, J. A., Pfeiffer, S. I., and LeBuffe, P. A. (1993). Devereux Behavior Rating Scale-School Form. San Antonio: The Psychological Corporation.

Naglieri, J. A., Pfeiffer, S. I., and LeBuffe, P. A. (1994). Devereux Scales of Mental Disorders. San Antonio: The Psychological Corporation.