PureGin.org

Main measurements are presented in Table 1.

Table 1. Mean scores for five major measures. (Standard deviations appear in parentheses.) Time measures are in seconds, Task Errors and Memory are percentage scores, and Subjective Satisfaction is on a scale from 1 to 10.

+ p < .10 * p < .05 ** p < .01 *** p < .001 (test for significant difference from control condition)

Hypothesis 1 was confirmed. Users of the scannable version performed tasks significantly faster than users of the control version did, t(19) = 1.95, p < .05, one-tailed. The same was true for users of the concise version, t(19) = 2.24, p < .05, one-tailed.

Hypothesis 2 was supported. Scannable users made significantly fewer task errors than control users, t(19) = 2.16, p < .05, one-tailed. Concise users also made fewer task errors, but the difference approached significance, t(19) = 1.47, p < .10, one-tailed.

Hypothesis 3 was confirmed. Scannable users had significantly better memory of site content than did control users, t(16) = -1.73, p < .05, one-tailed. Concise users did, as well, t(17) = -2.77, p < .01, one-tailed.

Hypothesis 4 was partially confirmed. As predicted, concise users took significantly less time to recall the site's structure than control users did, t(19) = 2.98, p < .001, one-tailed. However, there was no significant difference in the amount of time scannable users and control users took to remember the structure, t(19) = -0.40, p > .69.

As expected, there were no significant differences between the sitemap accuracy scores of the control users and: scannable users (t(19) = -0.16, p > .88), concise users (t(19) = -0.24, p > .82), or objective users (t(19) = -0.09, p > .93).

We did not predict (nor did we find) significant differences between objective users' and control users' measures for task time, task errors, memory, or sitemap time. However, compared to control users, objective users tended to perform the tasks faster, make fewer task errors, remember site content better, and recall the site structure faster. The differences are not significant, but they all point in the same direction (i.e., they suggest that the objective version is "better" than the control).

Hypothesis 5 was confirmed. Scannable users reported significantly higher subjective satisfaction with the site than control users did, t(19) = -2.41, p < .05, one-tailed . The same was true for concise users (t(19) = -1.85, p < .05, one-tailed) and objective users (t(19) = -1.76, p < .05, one-tailed).

Hypothesis 6 was confirmed. Users of the combined version performed tasks significantly faster than users of the control version did, t(19) = 3.30, p < .01, one-tailed. They also made fewer errors (t(19) = 3.36, p < .01, one-tailed), remembered more (t(17) = -4.56, p < .001, one-tailed), drew the sitemap faster (t(18) = 3.42, p < .01, one-tailed), and had higher subjective satisfaction (t(19) = -1.90, p < .05, one-tailed).

Hypothesis 7 was confirmed. Overall usability scores for all versions of the site show that, compared to the control version, the scannable version is 47% better, the concise version 58% better, the objective version 27% better, and the combined version was 124% better. Table 2 contains these data, as well as each condition's normalized mean score for each major measure. Nineteen out of 20 mean scores were higher than the corresponding scores for the control version, meaning that the other four versions were "better" than the control for nearly all of these measures.