Relationships between images are often of a sequential nature. Temporal sequences may include keyframes in an animation or frequently recorded satellite pictures. An example for spatial sequences is Magnetic Resonance Images (MRI) as they show successive slices of a volume. When interacting with these images, the user may wish to see detailed information without losing the context. Detail-in-context techniques provide methods to display parts of the data in full detail without sacrificing contextual information. Studies have shown that it is important to match the user's mental model as well as the underlying structure of the data when designing a detail-in-context algorithm. This paper describes a new algorithm to visualize sequential data and an application of this technique to the display of MR images.

Visualization, Detail-in-Context, Medical Imaging, MRI

The aim of detail-in-context visualization is to provide detailed information in a focused area, selected by the user, while maintaining context around that focus. We propose a detail-in-context technique that can be applied to data with sequential properties like time, as in video frames, or space, as in MR images. MRI scans are part of a spatial sequence since the scanned volume is sampled and viewed slice by slice. To present this information effectively, we have developed an algorithm that preserves horizontal alignment along data elements but unlike orthogonal detail-in-context layouts, breaks up vertical alignment to achieve better legibility.

The next section relates our work to the existing body of research. We then present the ideas that led to the design of our layout. The algorithm is then explained, examples are given, and limitations are discussed. In the last section, we briefly describe a user study that will be run in February 2000.

The idea of detail-in-context was introduced by Spence and Apperley’s "Bifocal Display" [6] and broadened in Furnas’ [2] "Generalized Fish-eye Views". Both visualize linear information, as well as the "Perspective Wall" [4] which offers a more advanced mapping of sequential data to screen space.

Despite the large amount of research that has been pursued in the visualization domain, current medical imaging systems still suffer from poor user interface designs [3]. Many existing PACS (Picture Archival and Communication System) use traditional zooming and scrolling to display medical images while others use thumbnails to select images for further magnification [1].

Van der Heyden et al. conducted an extensive analysis of MRI radiologists working with MR images in the traditional physical light screen environment. From this work, a set of requirements for the presentation of MRI volumes on a computer screen have been identified and a detail-in-context algorithm suggested [7]. This algorithm preserves orthogonality and minimizes image distortion. The research project presented in this paper begins with the algorithm identified by van der Heyden and extends it to achieve a better match with the data structure and with the mental model of our users.

While the Perspective Wall [4] provides an intuitive mapping of linear data to detail-in-context views, it is not sufficient if a large quantity of data must be displayed on the screen. With orthogonal layouts [5], the data stream is split and it is assumed that the user reads its elements from left to right and from top to bottom. An orthogonal layout was chosen in the presentation of MR images by van der Heyden et. al. [7] because this structure mapped well to the traditional method of displaying images on photographic films in horizontal and vertical rows.

However, orthogonal detail-in-context layouts have an inherent disadvantage. When multiple foci are selected that do not share the same row or column, so-called "ghost foci" occur as in Figure 1, i.e. nodes that are not selected become magnified due to orthogonal stretching. Even if the non-selected images remain small, the layout produces large unused spaces. This may be distracting for the user unless additional visual cues are provided. Furthermore, multiple foci are difficult to avoid in an MRI application given that radiologists often compare images that do not necessarily lie in the same row or column.

We developed an algorithm that creates a screen layout for sequential images, displaying detail while maintaining context, and uses the available screen space effectively. It also allows for comparison of multiple images, preserves the aspect ratio of the images, and matches the user's reading habits.

The main idea of our approach is to maintain horizontal but not vertical alignment. This decision was made because there is no meaningful relationship between images in the same column. Therefore, maintaining vertical alignment may be too constraining. In addition, this new layout does not produce ghost foci and, as a result, appears less distracting.

Our algorithm utilizes a one-dimensional array that contains all images. Every image that will appear magnified is set to the same fixed size to facilitate comparison between images. The array is then divided into multiple rows and the non-magnified images are resized to fit the row height and the window width. Every row contains the same number of images so they can be localized easily. This remains constant whether images are magnified or not. Figure 2 shows an example with two foci. The main disadvantage to this algorithm is that when many images in the same row are magnified, the surrounding images become very small.

Results of observations of radiologists using MRI films on a light screen support our layout method. Traditional films vary in the number of columns and are read in the same order that written text is read in Western cultures, that is, from left to right and from top to bottom. Thus, images in the same column are not related and therefore, vertical alignment is redundant. Horizontal alignment is maintained so that a row can be scanned with a single eye movement.

We have implemented a system that utilizes our algorithm to layout sequential images. We have used it to display MRI scans (spatially sequential data) as well as meteorological images (temporally sequential data).

A formal user study will be run in February 2000 to evaluate this proposed detail-in-context technique for the presentation of MR images. The experiment will involve radiologists as participants to compare the proposed algorithm with a typical layout utilized in most PACS. We will also compare this algorithm against the traditional light screen. Both quantitative and qualitative data will be gathered to examine the effectiveness of our approach for presenting and analyzing MR images. This study will also help to determine users' attitudes towards this new approach.

I would like to thank Dr. Kori Inkpen, Dr. Stella Atkins, Dr. M. Sheelagh Carpendale, and members of the EDGE Lab for their help and guidance in this project. Also, the help of the radiologists at Vancouver Hospital is greatly appreciated.