This article was downloaded by: [McGill University Library] On: 16 March 2015, At: 11:44 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Ergonomics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/terg20
The complex role of perceptual organization in visual display design theory a
a
PENELOPE M. SANDERSON , IAN HASKELL & JOHN M. FLACH
a
a
Department of Psychology and Department of Mechanical and Industrial Engineering , University of Illinois at Urbana-Champaign , 1206 West Green Street, Urbana, IL, 61801, USA Published online: 31 May 2007.
To cite this article: PENELOPE M. SANDERSON , IAN HASKELL & JOHN M. FLACH (1992) The complex role of perceptual organization in visual display design theory, Ergonomics, 35:10, 1199-1219, DOI: 10.1080/00140139208967390 To link to this article: http://dx.doi.org/10.1080/00140139208967390
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
ERGONOMICS, 1 9 9 2 , VOL
35,NO.
10,
I 199-1 2 19
The complex role of perceptual organization in visual display design theory PENELOPE M. SANDERSON', IAN HASKELL and JOHN M. F w 2 Depanment of Psychology and Depanment of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign, 1 206 West Green Street, Urbana, IL 61801, USA
Downloaded by [McGill University Library] at 11:44 16 March 2015
Keywords: Visual display design; Perceptual organization; Emergent features. Two experiments were performed to test and extend the current 'emergent features' approach to display design for dynamic failure detection tasks. It was predicted that a display whose emergent features were well-mapped to goalrelevant task invariants would lead to better failure detection performance than either of two alternative displays. Contrary to prediction, Experiment 1 showed no differences in failure detection speed or accuracy across displays. The reason for this may have been that subjects did not discover the mapping between the mathematical properties of the task and the emergent feature, so in Experiment 2 subjects were explicitly instructed about the mapping and were advised on how to use the display geometry to help their performance. A significant difference in failure detection speed emerged, but the display supporting fastest performance was not the one with the well-mapped emergent feature. These results suggest that alternative perceptual organizational factors were at work which overpowered the intended eftect. The results also underscore the difficulty of developing a theory of display design, and their impact on current theories is outlined. 1.
Introduction
1.1. Overview
Over the years there has been great interest in the design of static graphical displays, particularly since the rise of the statistical graphics movement (Cleveland 1 985, DeSanctis 1984, Tufte 1983, Lewandowsky and Spence 1988). There is also growing interest in the design of dynamic displays. The latter are often needed in technologyintensive domains, such as aviation and continuous process control, where the detection and interpretation of change over time is important (Beltracchi 1 986, Buttigieg and Sanderson 1 99 1, carswell and Wickens 1 987, Flach 1 WOa, 1WOb, Sanderson er a/. 1989, Vicente and Rasmussen 1 990, Woods and Elias 1988). In both areas the availability of inexpensive graphics power has removed many limitations on how data can be represented, and display design decisions are increasingly in the hands of software engineers without formal training in graphical design or visual perception. For neither static nor dynamic displays are there universally recognized principles for how data should be represented so that observers can e5tract the information they need in an efficient, error-free manner. Many sets of principles for display design have emerged but they suffer from either being extremely general, having an ambiguous scope of applicability, or not yet having clearly demonstrated effectiveness. However there are a growing number of attempts at providing a solid Correspondence about this paper should be sent to the first author at the address given. Now at Department of Psychology, Wright State University, Dayton, OH 45435, USA. 001441 39/9253.00 0 1992 Taylor & Francis Ltd.
1200
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
theory to back display design decisions. This paper will explore such an attempt for the design of dynamic displays. 1.2. Approaches to dynamic display design Over the last few years there have been two major schools of thought about how cognitive and perceptual psychology might be applied to the design of dynamic displays. One approach has drawn upon Garner's (1970) work on integral and separable dimensions and has proposed that performance at monitoring, failure detection and diagnosis will be most successful when the properties of a task match the properties of a stimulus. This is known as the 'compatibility of proximity' hypothesis (Boles and Wickens 1983, Carswell and Wickens 1987, Barnett and Wickens 1988, Wickens 1986). If a task requires mental integration ('high task proximity'), it is predicated that performance will be best when the data needed to perform the task are presented in an integral format ('high display proximity'). Conversely, if a task requires distinguishing separate elements ('low task proximity'), it i s predicated that performance will be best when the data are presented in a separated format ('low display proximity'). Tests of the compatibility of proximity hypothesis have been performed with both static and dynamic displays. Results compatible with the hypothesis have been found in some studies (Bamett and Wickens 1988, Wickens and Andre 1 990) but not in others (Coury and Purcell 1988, Coury et al. 1989, Sanderson et a!. 1989, Buttigieg and Sanderson 1991). One important reason for these failures to replicate has been the discovery of displays which, while ostensibly separated, actually support better integrated task performance than integral displays. These will be discussed below. A further problem with the theory has been the inability to develop precise definitions of display or task proximity (see discussion in Carswell and Wickens S 990a). In recent work, Carswell and Wickens ( 1990b) have emphasized that the homogeneity, or similarity, of the dimensions of a visual stimulus will increase its display proximity, making it more effective in supporting high proximity tasks. The compatibility of proximity approach has sought a match between general display properties and general hypothesized modes of human information processing. In contrast, an alternative approach defines tasks and stimuli in more concrete terms, and seeks the best mapping between moment-by-moment display geometry on the one hand, and states of the process that are relevant to the operator's task on the other (Flach 1988, 1990a, 1990b, Sanderson 1986, Sanderson er a/. 1989, Vicente and Rasmussen 1990, Woods and Elias 1988). The mapping is achieved not through an abstract classification of the display to be used and the task to be performed, but through a concrete understanding of the geometry of the display and the information the observer is seeking from the data. The focus is on providing operators with displays which provide obvious perceptual cues that allow them to 'see' directly in the display geometry the information they seek, rather than having them infer it cognitively. Proponents of both these approaches have seen the helpfulness of exploiting 'ernergen t features' or 'configural' properties of visual form when designing displays (Barnett and Wickens 1988, Carswell and Wickens 1987, Coury et al. 1989, Sanderson er al. 1989, Wickens and Andre 1990). These terms refer to the fact that when simple visual elements are combined, new perceptual features sometimes appear that were not in the simple visual elements. If, for example, three straight
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design lines are combined to form a triangle, two new perceptual features are closure and the area of the triangle. These features are called emergent features because they emerge from the arrangement without being present in its elements, and are very hard to ignore. A triangle would also be referred to as a configural display because the configuration of these elements produces emergent features. The terms 'emergent feature' and 'configurality' were introdticed by Gamer ( 1 970, 1 98 1 ), Pomerantz el al. (1977) and Treisman and Paterson (1984) in basic research on the perception of visual form. Recently, Sanderson et al. (1989) and Buttigieg and Sanderson ( 1 99 1 ) have tried to build a theoretical foundation for the second approach to dynamic display design discussed above by combining the idea of emergent features with the Gibsonian idea of'invariants' (Gibson 1979). An invariant is a fundamental and permanent property of a system which remains the same unless an abnormaIity is present. It arises through physical constraints in a system, and is usually expressible mathematically. It may be a parameter of a system, but it is bften a relation between several parameters. If such a parameter, or parameters, of a dynamic process is indeed expressible mathematically, and is mapped to the geometry of a visual form, the form will change when the process parameter changes. Going one step further, it should be possible to make the invariants of a process directly visible to an operator by mapping them to emergent features of the display. For this to be successful, the changes in an emergent feature over time must be distinct, as for instance when an enclosed shape moves from concavity to convexity or when a straight line appears to break. Thus emergent features must remain emergent over time just as much as over space. Moreover, the invariants of a system which are chosen for this mapping to emergent features must represent process properties that are crucial for the operator's task, whether it be monitoring, failure detection, or failure diagnosis. If this can be achieved, and its effectiveness demonstrated, then we have the foundation for a powerful theory of dynamic display design. Several experiments have provided support for this approach, which we will call the emergent features approach. One set of studies has shown that the emergent features approach predicts, and finds, display superiority effects quite contradictory to those predicted by Wickens' compatibility of proximity approach. For example, Coury and Purcell (1988) and Sanderson et al. (1989) have shown that integration task performance (which has high task proximity) can actually be better supported by a bar graph (which has low display proximity) than by an object display (which has high display proximity). This is because the bar graph, although a separated low proximity display, also has con figural properties emerging from the arrangement of the bars. Buttigieg and Sanderson (1991) have taken these results further to demonstrate that even when working with one type of display (bar graph, objects, etc.), a simple rearrangement of the way the process parameters are mapped to the display elements can lead to impressive display superiority effects. The motivation behind these rearrangements is to ensure that process invariants which are relevant to the subject's goals in the task are mapped onto emergent properties of the display. Under one arrangement the mapping can be more effective than the other. 1.3. Presen r research The, work presented in this paper was designed to extend the tradition of work examining the importance of emergent features and configurality effects for visual
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
display design. The goal was to show that the effective exploitation of emergent features can make integration tasks easier than they would be without such exploitation and to show the significance of this for visual display design. T o make the context of the present experiment coherent, it is necessary to describe in some detail the failure detection task whichawas used.. In much of the work performed to date, subjects have been required to monitor a continuous process and to detect failures. The process generally consists of two input parameters which mix together in some proportion to create an output parameter. The inputs are sine functions or sums of sine functions with frequencies well below 1 Hz. The general form of the equation relating the inputs (I, and I,) and the output (0) is as follows:
Figure 1 shows three alternative display formats for the input and output behaviour in equation 1 . In each case, the subject's impression is of perceptually continuous change as the bars move up and down according to their respective sine functions. Subjects monitor a display for failures of the equality in equation 1. Three general classes of failures have been used in prior research: ( I ) the equation becomes increasingly under- or over-additive; (2) a third, invisible, uncorrelated sine function has an increasing effect on the output, or (3) the coefficients a and b drift from their normal values. Perceptual features 'emerge' in these displays in differing degrees. Figure la shows a display in which the output bar is simply equidistant from the two inputs (the '101' display). Figure I b shows a display where the two inputs are placed beside each other and the output bar is placed to their right (the '110' display). These two displays were both used by Sanderson er al. ( 1 989) and Buttigieg and Sanderson (199 I ), and proved to support better failure detection performance than Carswell and Wickens' (1987) original triangle object display. Moreover, performance with the I01 display tended to be better than performance with the 110 display. This was particularly true when the coefficients a and b were equal (0.5 and 0.5).rather than unequal (0.7 and 0-3). This is because the I01 display shows a simple and, apparently, a highly salient emergent feature under normal conditions in the 0.510.5 condition: the tops of the bars are always lined up in a linearly ascending o r descending order. When the equality of equation 1 is violated, this linearity is broken, making the failure very easy to detect. However in the I10 display the normal and failed states of the process are not neatly indicated by linearity and nonlinearity, respectively, of the tops of the bars. The success of the I 0 1 display in the 0.510.5 weighting condition lay in the fact that an invariant of the process which camed the goal-relevant information was mapped onto an emergent feature of the visual form. In mathematical terms, the goal-relevant invariant was the first 'moment' of the process. When the invariant held, the bars lined up in linear fashion, and when the invariant was violated the linearity was also violated. In light of this, a possible explanation for the relatively worse performance with the I01 bar graph in the 0.7/0-3 weighting condition presents itself. In this condition, the output bar was no longer positioned at the moment of the process. Instead, it should have been positioned leftwards as shown in Figure lc, because the unequal weighting shifts the moment of the process towards the left. In the present experiment, we extend the emergent features approach by testing the hypothesis that if the 0.710.3 weighting is used in equation 1, and if the output of
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design
Figure 1.
.
The three displays used in Experiments 1 and 2. Figure l a is the 101-NMdisplay, figure l b the 110 display, and figure lc the 101-Mdisplay.
the bar graph is positioned at the moment of the process as in figure Ic, then failure detection performance will be better when subjects use the display shown in figure l c than when they use either of the other two displays. From now on we will refer to figure l a as the 101-NM (non-moment) display, figure lb as the 110 display, and figure I c as the 101-M (momen!) display. We predict that the 101-M display will support the fastest and most accurate failure detection performance because of the good mapping between process invariant and emergent feature. Subjects using the 101-NM display should show the next best performance, and those using the 1 1 0 display should d o worst of all. We chose a failure that would be difficult to detect, thus providing room for different displays to have a large potential effect on subjects' failure detection ability. (With a failure that is relatively easy to detect, a ceiling effect may hide display differences.) In the 'coefficient change' failure shown in equation 2, k varied from 0.7 (the value under normal conditions) to 0.1 and at the same time ( 1 -k) varied from 0-3 to 0-9.
This change in k caused the output to become less related to I , and more related to I *,
Downloaded by [McGill University Library] at 11:44 16 March 2015
1204
P. M. Sanderson et al.
but at no time did the output become higher than the higher input or lower than the lower input. In summary, in the two experiments to be reported we tested the effectiveness of the three display formats in failure detection tasks. Our hypothesis was that the IOI-M display should support most effective performance because an invariant of the process (output continually at the mathematical moment under normal conditions) had been mapped to a display emergent feature (linearity of the tops of the bars under normal conditions). The principal measure of the effectiveness of performance was failure detection latency and subjects were also asked to be as accurate as possible. In Experiment 1 the effectiveness of the three display formats was tested but, as will be seen, the results were not as predicted. In Experiment 2 the role of explicit instructions about how to exploit emergent features in the displays to perform the failure detection task was examined and this allowed a display effect to emerge. However this effect was not the one we expected.
2. Experiment 1 2. 1. Introduction In this experiment, performance at failure detection was tested with three visual displays: IOI-M, IOI-NM, and 110. If the emergent features hypothesis is sound, then reaction time should be shortest and accuracy highest for subjects working with the IOI-M display. 2.2. Method 2.2.1. Design: The design was a 3 x 3 mixed design. The two independent variables were session (within-subjects) and display type (between-subjects). Subjects were allocated to conditions on a rotating basis to minimize possible confounds arising from cohort history (Campbell and Stanley 1962). Subjects monitored two displays simultaneously, one on each side of a video screen (see figure 2). Each display represented a separate process. The actual displays used in the experiment have already been described in the Introduction. Subjects worked with the same type of display for three 50 min sessions. On the first day, subjects completed 13 3-min blocks. The first was a practice block and the remaining 12 were experimental blocks. On days 2 and 3, subjects always completed 12 3-min blocks only. At the end of day 3, subjects filled out a questionnaire and were debriefed.
Figure 2. An example of the experimental display that the subject worked with. The subject was required to monitor simultaneously two displays of the same type (in this example, the 101-M display).
2.2.2. System dynamics and jbilures: Normal process dynamics were given in equation 1. Each input value was a sum of two sine functions, with frequencies
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design
1205
chosen from a predefined set. The frequencies were chosen so that no input frequency was a harmonic of any other. At least 9 s elapsed after the beginning of each block before a failure occurred. Between 6 and 9 failures occurred at random intervals during a block. During the failure, the value of coefficient a of equation 1 moved smoothly from 0.7 to 0.1, and the value of coefficient b moved smoothly from 0-3to 0.9. Each failure continued until a key indicating its presence was pressed, or for I 5 s if no key was pressed, at which time the output was reset to its normal value. Simultaneous failures on the left and right hand process did not occur. The actual number of failures on each process were varied across blocks so that the subject did not expect a fixed number of them on any one block. Over the whole experiment, however, failures occurred almost equally often in the left and right hand process. The subjective impression of the display was of a set of bars smoothly moving up and down at different frequencies.
2.2.3. Apparatus: An IBM AT personal computer was used to present the displays and collect the data. The maximum height and width of the display were 7-62 crn and 5-2 cm, respectively. With a head position- 50 cm from the screen, these measurements subtend 8.66" and 5-94' of visual angle. 2.2.4. Procedure Subjects were told they would have to monitor and detect failures in two displays representing two processes. Details of the instructions given to subjects are provided below. Subjects were shown which keys to press to indicate a failure of the right or left hand process. The 'p' key was used to indicate failures on the right-hand process and the 'q' key for the left-hand process. There was auditory feedback associated with failure detection: a high tone meant that the subject had successfully identified the failure and a low tone signalled a false alarm or a missed failure. Subjects were also told that the output bar returned to its normal height after a failure was successfully detected or after 15 s, whichever came first. The experimenter instructed subjects to 'respond as quickly as possible while maintaining accuracy'. At the end of the experiment, subjects were asked to give an open-ended written report of the strategy or strategies they had used to perform the failure detection task. 2.2.5. Instructions: Subjects were told that in each process the output was the weighted average of the two inputs, and were shown equation 1 . They then saw a hard-copy picture of the display they would be required to monitor. Each bar in this picture was labelled with its name, but there were no additional markings on the display. The experimenter did not tell subjects anything about the expected overall appearance of the displays for normal operations or the possibility of exploiting emergent features to perform the task. (This reticence is important, as it will be contrasted with Experiment 2 with a situation where subjects were given fuller instructions).
2.2.6. Block performancejieedback: At the end of each block, subjects were shown a graph that summarized their accuracy for that block and for all previous blocks that day (see figure 3 for an example). The x axis showed accuracy, and the y axis'was average reaction time. Accuracy was defined as follows:
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
Accuracy =
1 0 0 [2 ~ x NF-(FA+M)] 2xNF
9
where FA is the number of false alarms, M is the number of failures not detected (misses),and N F is the number of failures. Subjects were told that their goal was to get as many points as possible in the upper right hand area of the graph, as marked by the lines in figure 3. This area represented fast responses and high accuracy. The number of failures, the number of detections, and the average reaction time were also provided in alphanumeric form. Subjects were encouraged to try to reach the goal area diagonally, by trying to increase accuracy and decrease RT simultaneously. If their accuracy was already above the level required they were encouraged to decrease RT, and vice-versa.
Figure 3. Block performance feedback graphic. Subjects were encouraged to achieve scores in the top right-hand box. Empty circles represent scores from previous trials and the filled circle represents the score from the immediately preceding trial. Accuracy was a weighted combination of false alarms and percent correct detections (see text for formula).
2.2.7. Subjccrs: There were 2 1 subjects in the experiment: eight seeing the 101-M display, seven the 101-NMdisplay, and six the 110 display. Subjects were paid $3.50 per session and there was also a $5.00 bonus for the subject in each cell of the design who performed the best. Approximately half the subjects were male and half were female. All subjects were students at the University of Illinois, some undergraduate and some graduate students. 2.3. Results Three dependent measures were taken at the end of each block; percentage of correct detections, the number of false alarms per experimental block, and time to detection ('reaction time'). A mixed between-within ANOVA was performed on these three dependent measures, with the between subjects factor of display and the withinsubjects factor of session. Homogeneity of variance tests were performed on all ANOVAs performed in this and the next experiment using Hartley's F ,, test (Kirk
Visual display design
' 0 - Experiornt l ..........
SECONDS 9
IOI-NM
0--.
........
...........
-.. ...
Downloaded by [McGill University Library] at 11:44 16 March 2015
7
'"A
-1
no
a
SESSIONS
Figure 4.
Reaction time in seconds for Experiments 1 and 2.
1968). Unless explicitly mentioned, all ANOVAs can be assumed to have met hornogenei ty of variance assumptions.
2.3.1. Reacrion time (RT): The results for RT are shown in figure 4. There was no effect of display or session, and no interaction between them (all Fs less than 1 a 0 ) . 2.3.2. Percentage of correct detections (PCD): The results for PCD are plotted in figure 5 . There was no effect of display, F(2,18)< 1. There was a significant improvement in performance across sessions, F(2,36)= 13.668, MSE = 1 36-92, p
Ergonomics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/terg20
The complex role of perceptual organization in visual display design theory a
a
PENELOPE M. SANDERSON , IAN HASKELL & JOHN M. FLACH
a
a
Department of Psychology and Department of Mechanical and Industrial Engineering , University of Illinois at Urbana-Champaign , 1206 West Green Street, Urbana, IL, 61801, USA Published online: 31 May 2007.
To cite this article: PENELOPE M. SANDERSON , IAN HASKELL & JOHN M. FLACH (1992) The complex role of perceptual organization in visual display design theory, Ergonomics, 35:10, 1199-1219, DOI: 10.1080/00140139208967390 To link to this article: http://dx.doi.org/10.1080/00140139208967390
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
ERGONOMICS, 1 9 9 2 , VOL
35,NO.
10,
I 199-1 2 19
The complex role of perceptual organization in visual display design theory PENELOPE M. SANDERSON', IAN HASKELL and JOHN M. F w 2 Depanment of Psychology and Depanment of Mechanical and Industrial Engineering, University of Illinois at Urbana-Champaign, 1 206 West Green Street, Urbana, IL 61801, USA
Downloaded by [McGill University Library] at 11:44 16 March 2015
Keywords: Visual display design; Perceptual organization; Emergent features. Two experiments were performed to test and extend the current 'emergent features' approach to display design for dynamic failure detection tasks. It was predicted that a display whose emergent features were well-mapped to goalrelevant task invariants would lead to better failure detection performance than either of two alternative displays. Contrary to prediction, Experiment 1 showed no differences in failure detection speed or accuracy across displays. The reason for this may have been that subjects did not discover the mapping between the mathematical properties of the task and the emergent feature, so in Experiment 2 subjects were explicitly instructed about the mapping and were advised on how to use the display geometry to help their performance. A significant difference in failure detection speed emerged, but the display supporting fastest performance was not the one with the well-mapped emergent feature. These results suggest that alternative perceptual organizational factors were at work which overpowered the intended eftect. The results also underscore the difficulty of developing a theory of display design, and their impact on current theories is outlined. 1.
Introduction
1.1. Overview
Over the years there has been great interest in the design of static graphical displays, particularly since the rise of the statistical graphics movement (Cleveland 1 985, DeSanctis 1984, Tufte 1983, Lewandowsky and Spence 1988). There is also growing interest in the design of dynamic displays. The latter are often needed in technologyintensive domains, such as aviation and continuous process control, where the detection and interpretation of change over time is important (Beltracchi 1 986, Buttigieg and Sanderson 1 99 1, carswell and Wickens 1 987, Flach 1 WOa, 1WOb, Sanderson er a/. 1989, Vicente and Rasmussen 1 990, Woods and Elias 1988). In both areas the availability of inexpensive graphics power has removed many limitations on how data can be represented, and display design decisions are increasingly in the hands of software engineers without formal training in graphical design or visual perception. For neither static nor dynamic displays are there universally recognized principles for how data should be represented so that observers can e5tract the information they need in an efficient, error-free manner. Many sets of principles for display design have emerged but they suffer from either being extremely general, having an ambiguous scope of applicability, or not yet having clearly demonstrated effectiveness. However there are a growing number of attempts at providing a solid Correspondence about this paper should be sent to the first author at the address given. Now at Department of Psychology, Wright State University, Dayton, OH 45435, USA. 001441 39/9253.00 0 1992 Taylor & Francis Ltd.
1200
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
theory to back display design decisions. This paper will explore such an attempt for the design of dynamic displays. 1.2. Approaches to dynamic display design Over the last few years there have been two major schools of thought about how cognitive and perceptual psychology might be applied to the design of dynamic displays. One approach has drawn upon Garner's (1970) work on integral and separable dimensions and has proposed that performance at monitoring, failure detection and diagnosis will be most successful when the properties of a task match the properties of a stimulus. This is known as the 'compatibility of proximity' hypothesis (Boles and Wickens 1983, Carswell and Wickens 1987, Barnett and Wickens 1988, Wickens 1986). If a task requires mental integration ('high task proximity'), it is predicated that performance will be best when the data needed to perform the task are presented in an integral format ('high display proximity'). Conversely, if a task requires distinguishing separate elements ('low task proximity'), it i s predicated that performance will be best when the data are presented in a separated format ('low display proximity'). Tests of the compatibility of proximity hypothesis have been performed with both static and dynamic displays. Results compatible with the hypothesis have been found in some studies (Bamett and Wickens 1988, Wickens and Andre 1 990) but not in others (Coury and Purcell 1988, Coury et al. 1989, Sanderson et a!. 1989, Buttigieg and Sanderson 1991). One important reason for these failures to replicate has been the discovery of displays which, while ostensibly separated, actually support better integrated task performance than integral displays. These will be discussed below. A further problem with the theory has been the inability to develop precise definitions of display or task proximity (see discussion in Carswell and Wickens S 990a). In recent work, Carswell and Wickens ( 1990b) have emphasized that the homogeneity, or similarity, of the dimensions of a visual stimulus will increase its display proximity, making it more effective in supporting high proximity tasks. The compatibility of proximity approach has sought a match between general display properties and general hypothesized modes of human information processing. In contrast, an alternative approach defines tasks and stimuli in more concrete terms, and seeks the best mapping between moment-by-moment display geometry on the one hand, and states of the process that are relevant to the operator's task on the other (Flach 1988, 1990a, 1990b, Sanderson 1986, Sanderson er a/. 1989, Vicente and Rasmussen 1990, Woods and Elias 1988). The mapping is achieved not through an abstract classification of the display to be used and the task to be performed, but through a concrete understanding of the geometry of the display and the information the observer is seeking from the data. The focus is on providing operators with displays which provide obvious perceptual cues that allow them to 'see' directly in the display geometry the information they seek, rather than having them infer it cognitively. Proponents of both these approaches have seen the helpfulness of exploiting 'ernergen t features' or 'configural' properties of visual form when designing displays (Barnett and Wickens 1988, Carswell and Wickens 1987, Coury et al. 1989, Sanderson er al. 1989, Wickens and Andre 1990). These terms refer to the fact that when simple visual elements are combined, new perceptual features sometimes appear that were not in the simple visual elements. If, for example, three straight
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design lines are combined to form a triangle, two new perceptual features are closure and the area of the triangle. These features are called emergent features because they emerge from the arrangement without being present in its elements, and are very hard to ignore. A triangle would also be referred to as a configural display because the configuration of these elements produces emergent features. The terms 'emergent feature' and 'configurality' were introdticed by Gamer ( 1 970, 1 98 1 ), Pomerantz el al. (1977) and Treisman and Paterson (1984) in basic research on the perception of visual form. Recently, Sanderson et al. (1989) and Buttigieg and Sanderson ( 1 99 1 ) have tried to build a theoretical foundation for the second approach to dynamic display design discussed above by combining the idea of emergent features with the Gibsonian idea of'invariants' (Gibson 1979). An invariant is a fundamental and permanent property of a system which remains the same unless an abnormaIity is present. It arises through physical constraints in a system, and is usually expressible mathematically. It may be a parameter of a system, but it is bften a relation between several parameters. If such a parameter, or parameters, of a dynamic process is indeed expressible mathematically, and is mapped to the geometry of a visual form, the form will change when the process parameter changes. Going one step further, it should be possible to make the invariants of a process directly visible to an operator by mapping them to emergent features of the display. For this to be successful, the changes in an emergent feature over time must be distinct, as for instance when an enclosed shape moves from concavity to convexity or when a straight line appears to break. Thus emergent features must remain emergent over time just as much as over space. Moreover, the invariants of a system which are chosen for this mapping to emergent features must represent process properties that are crucial for the operator's task, whether it be monitoring, failure detection, or failure diagnosis. If this can be achieved, and its effectiveness demonstrated, then we have the foundation for a powerful theory of dynamic display design. Several experiments have provided support for this approach, which we will call the emergent features approach. One set of studies has shown that the emergent features approach predicts, and finds, display superiority effects quite contradictory to those predicted by Wickens' compatibility of proximity approach. For example, Coury and Purcell (1988) and Sanderson et al. (1989) have shown that integration task performance (which has high task proximity) can actually be better supported by a bar graph (which has low display proximity) than by an object display (which has high display proximity). This is because the bar graph, although a separated low proximity display, also has con figural properties emerging from the arrangement of the bars. Buttigieg and Sanderson (1991) have taken these results further to demonstrate that even when working with one type of display (bar graph, objects, etc.), a simple rearrangement of the way the process parameters are mapped to the display elements can lead to impressive display superiority effects. The motivation behind these rearrangements is to ensure that process invariants which are relevant to the subject's goals in the task are mapped onto emergent properties of the display. Under one arrangement the mapping can be more effective than the other. 1.3. Presen r research The, work presented in this paper was designed to extend the tradition of work examining the importance of emergent features and configurality effects for visual
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
display design. The goal was to show that the effective exploitation of emergent features can make integration tasks easier than they would be without such exploitation and to show the significance of this for visual display design. T o make the context of the present experiment coherent, it is necessary to describe in some detail the failure detection task whichawas used.. In much of the work performed to date, subjects have been required to monitor a continuous process and to detect failures. The process generally consists of two input parameters which mix together in some proportion to create an output parameter. The inputs are sine functions or sums of sine functions with frequencies well below 1 Hz. The general form of the equation relating the inputs (I, and I,) and the output (0) is as follows:
Figure 1 shows three alternative display formats for the input and output behaviour in equation 1 . In each case, the subject's impression is of perceptually continuous change as the bars move up and down according to their respective sine functions. Subjects monitor a display for failures of the equality in equation 1. Three general classes of failures have been used in prior research: ( I ) the equation becomes increasingly under- or over-additive; (2) a third, invisible, uncorrelated sine function has an increasing effect on the output, or (3) the coefficients a and b drift from their normal values. Perceptual features 'emerge' in these displays in differing degrees. Figure la shows a display in which the output bar is simply equidistant from the two inputs (the '101' display). Figure I b shows a display where the two inputs are placed beside each other and the output bar is placed to their right (the '110' display). These two displays were both used by Sanderson er al. ( 1 989) and Buttigieg and Sanderson (199 I ), and proved to support better failure detection performance than Carswell and Wickens' (1987) original triangle object display. Moreover, performance with the I01 display tended to be better than performance with the 110 display. This was particularly true when the coefficients a and b were equal (0.5 and 0.5).rather than unequal (0.7 and 0-3). This is because the I01 display shows a simple and, apparently, a highly salient emergent feature under normal conditions in the 0.510.5 condition: the tops of the bars are always lined up in a linearly ascending o r descending order. When the equality of equation 1 is violated, this linearity is broken, making the failure very easy to detect. However in the I10 display the normal and failed states of the process are not neatly indicated by linearity and nonlinearity, respectively, of the tops of the bars. The success of the I 0 1 display in the 0.510.5 weighting condition lay in the fact that an invariant of the process which camed the goal-relevant information was mapped onto an emergent feature of the visual form. In mathematical terms, the goal-relevant invariant was the first 'moment' of the process. When the invariant held, the bars lined up in linear fashion, and when the invariant was violated the linearity was also violated. In light of this, a possible explanation for the relatively worse performance with the I01 bar graph in the 0.7/0-3 weighting condition presents itself. In this condition, the output bar was no longer positioned at the moment of the process. Instead, it should have been positioned leftwards as shown in Figure lc, because the unequal weighting shifts the moment of the process towards the left. In the present experiment, we extend the emergent features approach by testing the hypothesis that if the 0.710.3 weighting is used in equation 1, and if the output of
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design
Figure 1.
.
The three displays used in Experiments 1 and 2. Figure l a is the 101-NMdisplay, figure l b the 110 display, and figure lc the 101-Mdisplay.
the bar graph is positioned at the moment of the process as in figure Ic, then failure detection performance will be better when subjects use the display shown in figure l c than when they use either of the other two displays. From now on we will refer to figure l a as the 101-NM (non-moment) display, figure lb as the 110 display, and figure I c as the 101-M (momen!) display. We predict that the 101-M display will support the fastest and most accurate failure detection performance because of the good mapping between process invariant and emergent feature. Subjects using the 101-NM display should show the next best performance, and those using the 1 1 0 display should d o worst of all. We chose a failure that would be difficult to detect, thus providing room for different displays to have a large potential effect on subjects' failure detection ability. (With a failure that is relatively easy to detect, a ceiling effect may hide display differences.) In the 'coefficient change' failure shown in equation 2, k varied from 0.7 (the value under normal conditions) to 0.1 and at the same time ( 1 -k) varied from 0-3 to 0-9.
This change in k caused the output to become less related to I , and more related to I *,
Downloaded by [McGill University Library] at 11:44 16 March 2015
1204
P. M. Sanderson et al.
but at no time did the output become higher than the higher input or lower than the lower input. In summary, in the two experiments to be reported we tested the effectiveness of the three display formats in failure detection tasks. Our hypothesis was that the IOI-M display should support most effective performance because an invariant of the process (output continually at the mathematical moment under normal conditions) had been mapped to a display emergent feature (linearity of the tops of the bars under normal conditions). The principal measure of the effectiveness of performance was failure detection latency and subjects were also asked to be as accurate as possible. In Experiment 1 the effectiveness of the three display formats was tested but, as will be seen, the results were not as predicted. In Experiment 2 the role of explicit instructions about how to exploit emergent features in the displays to perform the failure detection task was examined and this allowed a display effect to emerge. However this effect was not the one we expected.
2. Experiment 1 2. 1. Introduction In this experiment, performance at failure detection was tested with three visual displays: IOI-M, IOI-NM, and 110. If the emergent features hypothesis is sound, then reaction time should be shortest and accuracy highest for subjects working with the IOI-M display. 2.2. Method 2.2.1. Design: The design was a 3 x 3 mixed design. The two independent variables were session (within-subjects) and display type (between-subjects). Subjects were allocated to conditions on a rotating basis to minimize possible confounds arising from cohort history (Campbell and Stanley 1962). Subjects monitored two displays simultaneously, one on each side of a video screen (see figure 2). Each display represented a separate process. The actual displays used in the experiment have already been described in the Introduction. Subjects worked with the same type of display for three 50 min sessions. On the first day, subjects completed 13 3-min blocks. The first was a practice block and the remaining 12 were experimental blocks. On days 2 and 3, subjects always completed 12 3-min blocks only. At the end of day 3, subjects filled out a questionnaire and were debriefed.
Figure 2. An example of the experimental display that the subject worked with. The subject was required to monitor simultaneously two displays of the same type (in this example, the 101-M display).
2.2.2. System dynamics and jbilures: Normal process dynamics were given in equation 1. Each input value was a sum of two sine functions, with frequencies
Downloaded by [McGill University Library] at 11:44 16 March 2015
Visual display design
1205
chosen from a predefined set. The frequencies were chosen so that no input frequency was a harmonic of any other. At least 9 s elapsed after the beginning of each block before a failure occurred. Between 6 and 9 failures occurred at random intervals during a block. During the failure, the value of coefficient a of equation 1 moved smoothly from 0.7 to 0.1, and the value of coefficient b moved smoothly from 0-3to 0.9. Each failure continued until a key indicating its presence was pressed, or for I 5 s if no key was pressed, at which time the output was reset to its normal value. Simultaneous failures on the left and right hand process did not occur. The actual number of failures on each process were varied across blocks so that the subject did not expect a fixed number of them on any one block. Over the whole experiment, however, failures occurred almost equally often in the left and right hand process. The subjective impression of the display was of a set of bars smoothly moving up and down at different frequencies.
2.2.3. Apparatus: An IBM AT personal computer was used to present the displays and collect the data. The maximum height and width of the display were 7-62 crn and 5-2 cm, respectively. With a head position- 50 cm from the screen, these measurements subtend 8.66" and 5-94' of visual angle. 2.2.4. Procedure Subjects were told they would have to monitor and detect failures in two displays representing two processes. Details of the instructions given to subjects are provided below. Subjects were shown which keys to press to indicate a failure of the right or left hand process. The 'p' key was used to indicate failures on the right-hand process and the 'q' key for the left-hand process. There was auditory feedback associated with failure detection: a high tone meant that the subject had successfully identified the failure and a low tone signalled a false alarm or a missed failure. Subjects were also told that the output bar returned to its normal height after a failure was successfully detected or after 15 s, whichever came first. The experimenter instructed subjects to 'respond as quickly as possible while maintaining accuracy'. At the end of the experiment, subjects were asked to give an open-ended written report of the strategy or strategies they had used to perform the failure detection task. 2.2.5. Instructions: Subjects were told that in each process the output was the weighted average of the two inputs, and were shown equation 1 . They then saw a hard-copy picture of the display they would be required to monitor. Each bar in this picture was labelled with its name, but there were no additional markings on the display. The experimenter did not tell subjects anything about the expected overall appearance of the displays for normal operations or the possibility of exploiting emergent features to perform the task. (This reticence is important, as it will be contrasted with Experiment 2 with a situation where subjects were given fuller instructions).
2.2.6. Block performancejieedback: At the end of each block, subjects were shown a graph that summarized their accuracy for that block and for all previous blocks that day (see figure 3 for an example). The x axis showed accuracy, and the y axis'was average reaction time. Accuracy was defined as follows:
P. M. Sanderson et al.
Downloaded by [McGill University Library] at 11:44 16 March 2015
Accuracy =
1 0 0 [2 ~ x NF-(FA+M)] 2xNF
9
where FA is the number of false alarms, M is the number of failures not detected (misses),and N F is the number of failures. Subjects were told that their goal was to get as many points as possible in the upper right hand area of the graph, as marked by the lines in figure 3. This area represented fast responses and high accuracy. The number of failures, the number of detections, and the average reaction time were also provided in alphanumeric form. Subjects were encouraged to try to reach the goal area diagonally, by trying to increase accuracy and decrease RT simultaneously. If their accuracy was already above the level required they were encouraged to decrease RT, and vice-versa.
Figure 3. Block performance feedback graphic. Subjects were encouraged to achieve scores in the top right-hand box. Empty circles represent scores from previous trials and the filled circle represents the score from the immediately preceding trial. Accuracy was a weighted combination of false alarms and percent correct detections (see text for formula).
2.2.7. Subjccrs: There were 2 1 subjects in the experiment: eight seeing the 101-M display, seven the 101-NMdisplay, and six the 110 display. Subjects were paid $3.50 per session and there was also a $5.00 bonus for the subject in each cell of the design who performed the best. Approximately half the subjects were male and half were female. All subjects were students at the University of Illinois, some undergraduate and some graduate students. 2.3. Results Three dependent measures were taken at the end of each block; percentage of correct detections, the number of false alarms per experimental block, and time to detection ('reaction time'). A mixed between-within ANOVA was performed on these three dependent measures, with the between subjects factor of display and the withinsubjects factor of session. Homogeneity of variance tests were performed on all ANOVAs performed in this and the next experiment using Hartley's F ,, test (Kirk
Visual display design
' 0 - Experiornt l ..........
SECONDS 9
IOI-NM
0--.
........
...........
-.. ...
Downloaded by [McGill University Library] at 11:44 16 March 2015
7
'"A
-1
no
a
SESSIONS
Figure 4.
Reaction time in seconds for Experiments 1 and 2.
1968). Unless explicitly mentioned, all ANOVAs can be assumed to have met hornogenei ty of variance assumptions.
2.3.1. Reacrion time (RT): The results for RT are shown in figure 4. There was no effect of display or session, and no interaction between them (all Fs less than 1 a 0 ) . 2.3.2. Percentage of correct detections (PCD): The results for PCD are plotted in figure 5 . There was no effect of display, F(2,18)< 1. There was a significant improvement in performance across sessions, F(2,36)= 13.668, MSE = 1 36-92, p
