Test table for usability tasks

Cochran's Q statistic is a chi-square variate formed by a ratio of the variation in success across tasks to the variation in success within subjects. Based on the statistics and frequency tables, you would expect a large statistic because you observed quite a bit of variation in success by task.

  • Degrees of freedom for this chi-square are equal to the number of test variables minus 1. There were six tasks, so there are five degrees of freedom.
  • The asymptotic significance is the approximate probability of obtaining a chi-square statistic as extreme as 12.949 in repeated samples if the frequencies of task success are only randomly different.

Because a chi-square this large is unlikely to have arisen by chance, the design team rejects the null hypothesis that all tasks have an equal number of successes. Clearly, users had difficulty interacting with the support list, as well as the fax and newsletter request pages.