CREATING A BEHAVIORALLY ANCHORED RATING SCALE

The main purpose of this group project is to illustrate how to develop a performance measure for a job, which distinguishes between successful job performance and unsuccessful job performance. Your task is to develop a behaviorally anchored rating scale to measure the performance of your team members on a group project at school. All of you have worked on a group project at one point in time, and you realize the difficulties that may be experienced in this particular group setting. Here is your chance to develop your own peer evaluation, indicating the areas you believe are important to effective group behavior. Each of you in the group will be required to contribute to the completion of your group evaluation.

Step 1: Decide upon the critical job dimensions you wish to measure (general areas of performance). What areas are important to the performance of a student in a group project situation? For example, a recruitment specialist's job dimensions may be: interviewing effectiveness, verbal communication, interpersonal abilities, etc. You may wish to each develop your own list and then discuss the dimensions as a group, or to create the list through group discussion.

Step 2: Each person in the group should then independently develop a series (at least ten per group member) of statements (behaviors) that describe positive, negative and average behavior or outcomes for each job dimension. These should be specific behaviors that can be measured/recognized by an appraiser. In this situation, the appraiser will be the other members of your group. For example, the following statements might be developed for the interviewing effectiveness function of a recruitment specialist:

• Puts the recruit at ease at the beginning of the interview.
• Has reviewed the job description and job specification prior to the interview.
• Has reviewed the application and resume of the recruit prior to the interview.
• Utilizes open-ended questions.
• Probes for more information to gain insight into the recruit.

Frequently fails to take notes and complete the rating process after an interview is complete. Often asks questions that are not job related.

REMEMBER: YOU MUST HAVE POSITIVE, NEGATIVE AND AVERAGE BEHAVIORS FOR EACH JOB DIMENSION.

Step 3: After each group member has developed a series of statements for each dimension, the statements should be reviewed by each group member to assess the dimensions importance and to decide which dimension is described by the statement. To do this you should:

1. Write each statement on a single index card
2. Have each group member assess the level of importance for each statement as an example of good, bad or average performance. Use a 1 to 7 scale with 7 indicating the highest level of importance. The point of this step is to assess whether the statement is important enough to be included in your scales. Do not try to organize the behaviors into a scale, or decide whether the behavior is good, bad or average, you will do this later. Simply decided if the statement describes a behavior or outcome important to group member behavior. A seven implies that the statement is very important to group member behavior. A one implies that the statement is not important. Look at the behavior on the font of the card, decide its level of importance and record you score on the back of the care. Do not read the scores recorded by other group members prior to making your decision, as to not bias your own decision.
3. Each person must also decide which critical job dimension the statement represents. Look at the behavior on the front of the card, decide under which category the behavior belongs and record the category on the back of the card. As discussed above, do not read the responses of other group members prior to making your decision.
4. Compute the mean and standard deviation for the scores recorded on each item. For each statement, you should have an importance score assigned by each group member. These are the scores you should use to compute the mean and standard deviations.
5. Discard items for which there is a disagreement concerning the importance of the item (i.e. large standard deviation) or items that are deemed unimportant (i.e. small mean). Also if your group disagrees on the dimension under which the statement belongs, reject the item.
6. Develop your scales by using the best items to anchor them. Remember you need examples of good, bad and average behavior for each dimension. Put a scale together for each dimension using the behaviors remaining after completing the above decision process.
7. Weight the importance of each dimension or category so cumulative scores can be calculated. Explain your rationale for assigning the weights. Why do you feel one category of behaviors is more important to performance than another category of behaviors?

Step 4: Your final report should include the following:

1. A list of all behavioral statements with their corresponding mean, standard deviation, and dimension (include even those that were rejected).
2. A summary sheet detailing and explaining the rationale used to select your dimensions.
3. An explanation of the criteria your group used to reject and utilize behavioral statements (what cut offs did you use and why?).
4. A BARS scale for each dimension of work performance.