Test Developer Profiles

Ann E. Boehm, Ph.D.

Test Developed:

The Boehm Test of Basic Concepts

When searching for a dissertation topic, I became interested in why some children met greater success than others in kindergarten and first grade. The "why" moved beyond the child's cognitive abilities, and I concentrated on the expectations that teachers had and the instructional materials encompassed. A review of the literature revealed a lack of research regarding the conditions associated with the attainment of concepts involved in school learning during the early grades. Through interviews with teachers and review of curricular materials used in kindergarten through grade 3 in reading and mathematics, it became clear that a hidden curriculum could underlie children's ability to be successful with many tasks presented during their early schooling. In order to follow teacher directions and comply with instructional activities, children needed to know relational concepts such as "top," "begin," "row," and so forth. For example, to comply with a typical teacher direction such as "Mark all of the a's' in the top row," the child needs to know not only the letter "a," but also the concepts, all, top, and row. A review of curricular materials in math and reading across publishers resulted in the identification of the target concepts. The Test of Comparative Concepts (now the Boehm Test of Basic Concepts) was devised to determine whether relational concepts commonly used in primary-school curricular materials and teacher directions, often assumed present at the time of school entrance, were possessed by young children. The concepts measured were chosen because of the frequency with which they were used in primary curricula, particularly in directions given to the pupil, and the relative lack of emphasis within standard textbooks and workbooks for developing these same concepts.

The results of the test administered to 1,283 pupils attending grade K through grade 3, showed that the development of comparative concepts was related not only to grade/age but also to background variables, including parental occupation. Children at the end of third grade from lower SES backgrounds performed like children at the end of kindergarten programs from middle SES backgrounds. Based on their basic concept knowledge, children from more advantaged backgrounds came to school better prepared to meet the demands of instruction (Boehm, 1966).

My dissertation sponsor (Mary Alice White) believed the test to be of publishable quality and put me in touch with the appropriate individuals at The Psychological Corporation. I also was fortunate to have Robert Thorndike and Milly Almy, well-known scholars in the areas of measurement and child development, as members of my committee; they gave me excellent guidance. I was fortunate as well to have support from my internship setting (the Devereaux Schools) for data collection. Not only did their sheltered workshop run off and collate more than 1,300 copies of the test, my supervisor (William Cohen) helped me locate representative schools in which to conduct the study. Influences on Defining the Construct of Basic Concepts

My thinking about basic concept acquisition has been influenced by the work of researchers such as Piaget, Gagne, Klausmeier, Vygotsky, Carroll, Bruner, French and Nelson, de Villiers and de Villiers, and both Herbert Clark and Eve Clark, among many others. The research literature that has emerged since the BTBC was first developed has focused on how individual relational concepts develop, the cognitive strategies children use to solve problems, the meaning of errors, and the effects of scaffolding on concept acquisition. I have found that children as young as 3 can provide reasons for their responses if one asks questions they can respond to. This body of research was important in developing the Boehm Resource Guide for Basic Concept Teaching and continues to guide my thinking today.

Noteworthy Aspects of Test Development

Working with a major publisher has provided me with an understanding of how complex the test development process is, including the importance of (a) an ongoing comprehensive review of the research literature, as well as documentation of outcomes of a newly developed or revised test; (b) field testing with concepts represented through good artwork and using alternative representations of concepts prior to the selection of the final items to be included during standardization; (c) a representative standardization sample; (d) a review of potential bias at the beginning of each test revision; (e) input from users of the test; (f) the ongoing need to engage in reliability and validity studies on the part of the author and other researchers; and (g) the need to develop materials that are useful when working with children who present different handicapping conditions or who come from different ethnic and language backgrounds. In the case of the Boehm III (now in process), this will include children with learning difficulties, a sensory handicap, and children from different language backgrounds.

Pros and Cons of Being a Test Developer

Being a test developer has been exciting for me. It certainly keeps me up-to-date with the related research literature. The pros for me have been working with a major test publisher, which has many advantages including the test publisher providing art, locating sample populations, providing materials, providing data analysis, and engaging in research–"all in concert with the author. The cons include (a) the long timeframe and detail involved in test development and revision (test development involves a comprehensive process that takes time, patience, and tenacity), and (b) as I have learned over the years, the validity and reliability studies reported in test manuals and presented at professional meetings associated with developing a test and with keeping it up-to-date are not necessarily recognized as serious empirical research. Advice to prospective test developers: Submit articles involving research with your test to refereed journals on an ongoing basis. And, use the comments of reviewers to better your instrument.

References

Boehm, Ann E. (1966). The development of comparative concepts in primary school children. Unpublished doctoral dissertation, Teachers College, Columbia University, New York.