Test Developer Profiles

Kenneth W. Merrell, Ph.D.

Tests Developed:

School Social Behavior Scales

Preschool and Kindergarten Behavior Scales

Internalizing Symptoms Scale for Children (Merrell & Walters, 1998)

Major Influences on Test Development

It is difficult to pinpoint exactly what were my major influences on test development, because they were many. While working as a practicing school psychologist early in my career, I used a variety of assessment instruments with children and their families and was particularly interested in assessing social and emotional behavior of children and adolescents. Some of these instruments were most useful, whereas others had numerous problems or other limitations. I think that my observation of the limitations of existing measures, coupled with my desire to improve the state of the art, led me in the direction of becoming a test developer.

Aside from these general influences, there was a very specific and strong influence on my early work that led to the development of the School Social Behavior Scales (Merrell, 1993) and the Preschool and Kindergarten Behavior Scales (Merrell, 1994). My doctoral dissertation at the University of Oregon involved identifying multiple influences and predictor variables that were associated with child study team decisions regarding whether or not a referred student should be classified as having a learning disability. I used a variety of behavioral, cognitive, and academic measures in this study. One of the instruments I used was an experimental pre-publication version of the Walker-McConnell Scale of Social Competence and School Adjustment (Walker & McConnell, 1995). Hill Walker, the senior author of this test, was a professor in my college and became an ad hoc member of my dissertation committee. I remember being extremely impressed with not only the care and detail that had gone into developing this test but also the practical, useful nature of the final product. Within a couple of years after my graduation, I determined that I wanted to develop a test (the School Social Behavior Scales) wherein I could model some of the fine work I had seen in the Walker-McConnell Scale, but that would also make a unique contribution and not merely be a copycat test (the School Social Behavior Scales were designed specifically to assess both the constructs of social skills and antisocial behavior). Before embarking on this venture, I met with Hill Walker to discuss my idea with him. He was very gracious and encouraging to me. As a researcher who was very young and naive at the time, I simply don't think I would have proceeded without Hill's support and encouragement, something for which I will always be grateful.

Influences in Defining the Constructs of Social Skills and Antisocial Behavior

In developing both the School Social Behavior Scales and the Preschool and Kindergarten Behavior Scales, my work was theoretically influenced by Walker's theories regarding types of social adjustments that children make (e.g., Walker & McConnell, 1995) and by Gerald Patterson's theories regarding the development of antisocial behavior in children (e.g., Patterson, Reid, & Dishion, 1992). In addition, my work has always been strongly influenced or informed by social learning theory, pioneered by Albert Bandura (e.g., Bandura, 1986). Based on these influences, I maintain that social competence and social skills are complex multidimensional constructs and that the development of both positive and negative social behavior is a reciprocal process that has many influences and that ultimately shapes future behavior. I also maintain that although children who have high levels of social skills are likely to have low levels of antisocial behavior or other problem behaviors, the two constructs are not merely polar opposites of a continuum. They are certainly related constructs, but they operate independently.

Interesting and/or Noteworthy Aspects of the Test Publication Process

What I have found to be extremely reinforcing about the text development and publication process is identifying a need and then developing something that you actually see meeting that need. For example, when I developed the Preschool and Kindergarten Behavior Scales, I clearly saw a need for a practical social skills and problem behavior screening test for use by parents and teachers in evaluating "garden variety" child behavior concerns. As I began to see this test used in Head Start programs, developmental preschools, and child find screenings around the nation and began to receive positive feedback from the users, it was tremendously reinforcing to me and made the process very worthwhile.

One of the most noteworthy aspects of the test publication process is how difficult it has become to find a commercial publisher to take a risk on a new test. I am confident that test developers who start a project from scratch and then shop for a publisher are in for a lot of work, discouragement, and a long haul in general. My observation is that test development has become an increasingly competitive and commercialized venture since the 1970s, and much of it is done in a self-contained manner at the corporate level now. The amount of resources that go into marketing alone for certain types of tests is astounding, and thus requires that sales be commensurately large enough to justify such marketing. I predict that there will be fewer and fewer independent test developers who see a need, develop an idea, produce a product, and then find a publisher to market it. I think that for better or worse, educational and psychological tests will increasingly be conceptualized, developed, and produced within a self-contained process by large commercial test publishers, and that independent researchers and "mom and pop" research and development operations will find it increasingly difficult to compete or find a market for their work. I have been surprised by the difficulty that some of my colleagues, who are nationally known for their work, have recently had in finding a home for some very good test products.

Perspective: The Pros and Cons of Being a Test Developer

To me, the best thing about being a test developer is the great sense of satisfaction that only comes through carrying out a complex project from conceptualization to final production and dissemination. Of course, external validation of your work is important. Seeing your test being used and well received by the professional community is a tremendous reinforcer, as is seeing positive reviews of your work in the professional literature. On the other hand, putting your work out in front of the world opens up a lot of room for criticism, so you need to be prepared to take some heat. In other words, don't take it too personally when the inevitable time comes when someone says something critical about your work. If your work is generating some flak as well as praise, it is a sign that it is considered important enough to justify comment. To me, this scenario is much better than having your work ignored. Remember that one does not need to have any credentials to offer criticism. I think that authors would rather have their work critiqued by those whom they truly consider peers, in other words, people who have been through the process themselves, who cannot only "talk the talk" but "walk the walk." However, this is wishful thinking. When you produce a book or a test for large-scale use, you enter the democratic public domain where everyone is entitled to their opinion, regardless of what contributions they have made, and where others' opinions may not always coincide with yours.

Advice to Prospective Test Developers

In my view, the most important advice to prospective test developers is to put in whatever effort and resources are required up front to do the job right, even if it results in the process taking longer than desired or anticipated. The most important place to start the job right is in defining the construct to be measured and coming up with an appropriate strategy and items for measuring it. It is impossible to overemphasize how important it is to do the item development phase well and to go through a thorough and detailed content validation process before any standardization data are ever gathered. Doing the job well in the early planning and development stages will make your work infinitely easier later on and will certainly result in stronger psychometric properties for your test. Consider using an expert content validation panel, as well as focus groups of prospective test subjects. For example, if your test is going to be targeted toward parents of preschool-age children, it is important to solicit feedback from people of this demographic regarding your test prototype before you actually begin to collect standardization data or conduct psychometric studies.

The preceding advice was general, based on the idea that there are many reasons for developing tests, ranging from research to commercial use. If you want to produce a test that will become commercially published, my advice is that it is necessary to take the development and planning process even further. First, you should conduct a thorough needs assessment to determine if there truly is a commercial need or potential market for what you want to produce. Part of this needs assessment must necessarily involve identifying what tests are currently available related to what you are trying to do and ensuring that your measure is somehow different and offers some advantages that the existing tests do not offer. Second, unless you want to go into the business of publishing and disseminating your test by yourself, which is clearly a daunting task, I recommend that you start to make contacts and cultivate relationships with test publishing companies early in the process.

A final note regarding the amount of work that goes into developing a test the right way: I have developed three nationally standardized norm-referenced tests to date and have written the same number of books. In comparing the two processes, I consider writing a book and getting it published to be a much easier venture!

References

Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice-Hall.

Merrell, K. W. (1993). School Social Behavior Scales. Austin, TX: Pro-Ed.

Merrell, K. W. (1994). Preschool and Kindergarten Behavior Scales. Austin, TX: Pro-Ed.

Merrell, K. W., & Walters, A. S. (1998). Internalizing Symptoms Scale for Children. Austin, TX: Pro-Ed.

Patterson, G. R., Reid, J., & Dishion, T. (1992). Antisocial boys. Eugene, OR: Castalia.

Walker, H. M., & McConnell, S. R. (1995). Walker-McConnell Scale of Social Competence and School Adjustment: Elementary Version. San Diego, CA: Singular Publishing Group.