Conformal Magnifier: A Focus+Context Technique with Minimal Distortion.

HHS Public Access Author manuscript Author Manuscript

Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14. Published in final edited form as: Visualization (Los Alamitos Calif). 2012 November ; 18(11): 1928–1941.

Conformal Magnifier: A Focus+Context Technique with Minimal Distortion Xin Zhao, Wei Zeng, Xianfeng Gu, Arie Kaufman, Wei Xu, and Klaus Mueller Stony Brook University

Abstract Author Manuscript

We present the conformal magnifier, a novel interactive Focus+Context visualization technique to magnify a region of interest (ROI) using conformal mapping. Our framework allows the user to design an arbitrary magnifier to enlarge the features of interest while deforming part of the remaining areas without any cropping. By using conformal mapping, the ROI is magnified with minimal distortion, while the transition region is a smooth and continuous deformation between the focus and context regions. An interactive interface is designed for the user to select important features, design focus models of arbitrary shape and set deformation constraints to satisfy his/her specified requirements. We demonstrate the effectiveness, robustness and efficiency of our method using several applications: texts, maps, geographic images, data structures and multimedia visualization.

Author Manuscript

Keywords Focus+Context technique; Conformal mapping; Information visualization

1 Introduction

Author Manuscript

Visualization can greatly help professionals in the understanding of data in science, engineering, medicine, business, and finance. With the emergence of highly visual mainstream applications as diverse as google maps, internet photo collections, MS Excel, and others, interactive visualization has also become a popular and increasingly familiar paradigm for a more general user base. With the tremendous increases in computing power, data storage, and internet bandwidth, great advances have been made for all types of visualization applications, including maps, satellite images, charts, multi-media, and the like. We can now easily store, process, and deliver over the internet very large data sets, but an inherent limitation is the amount of space available to display these data. While display devices may have grown in size and resolution, a natural limit is and remains to be the human's visual field of view. At the same time, with the emergence of portable devices, such as netbooks and smart phones, there has also been a reverse trend in screen size for mobile [email protected]. [email protected]. [email protected]. [email protected]. [email protected]. [email protected].

Zhao et al.

Page 2

Author Manuscript Author Manuscript

applications. So no matter what display size is being used, a careful management of the display real estate is direly required - as a general rule, no pixel should go wasted with the display of redundant or irrelevant information. A natural solution to these requirements is Focus+Context (F+C) visualization. It allows viewers to focus on the subset of items which they are most interested in, but within the overall setting of these data. Such visualizations allow users to grasp the overall theme of the data to guide the sense-making process, allowing them to access and address the appropriate internalized knowledge “bins”. Many F +C techniques have become available in recent years. As a general rule, a good F+C method must support continuity when transitioning from the magnified to the minified areas. Only then can the user perform effective visual search at these multiple levels of scales. Such transitions are natural to humans because it matches their biological visual systems where peripheral vision has less detail than the retinal areas closer to the fovea centralis. Optical lenses have been available for centuries and humans have become very familiar with them. Computers can easily simulate their effects, but they also provide great opportunity to overcome their limitations and also to implement what has been often referred to as magic lenses: lenses that can reveal detail in ways not possible with physical devices.


Our paper addresses a specific limitation that optical lenses as well as their many digital counterparts have: their local distortion. Since humans have become accustomed to lens distortions over centuries of use, they have come to accept them and even mentally compensated for their effects. Take, for example, the surgeon who performs endoscopic minimally invasive procedures aided by lens devices day in day out. He mentally unwarps the presented imagery without problems. He actually would find undistorted views unfamiliar and harder to reason with. This is because the patterns in this working environment are of a scale that can tolerate these distortions and there is also the motion parallax that emerges when the lens is moved helping to assimilate these shapes. However, when features become sufficiently intricate, and when this motion parallax is missing as in static presentational views then lens distortions are intolerable. An example for more mainstream applications is text. While the pattern recognition system of humans can deal with noisy and distorted text quite easily, given sufficient literacy skills, this skill rapidly degrades when the text is in a foreign language, contains unfamiliar terms, or has subtle and ambiguous meanings. The same goes for intricate and unknown detail in information as well as medical and scientific data displays. While current lenses are “magic” in the sense what they can reveal, they are less so when it comes to overcoming distortion effects, call them artifacts, which are rooted in their lack of awareness of these. Distortion is commonly measured in entropy-like metric, that is, they gauge how much information one can pack in a certain magnified (or minified) screen area. We address, in our paper, the specific need to control local distortion, to preserve local detail undistorted and so enable user to reliably “read” and decode them. Our lens uses the concept of conformal mapping and zooming to capture the F+C regions of interest into a single view while providing smooth transition between the F+C regions. Through our interface, users can design arbitrary lens shape models to act as the magnifier and set constraints for its region of interest (ROI). A suite of 1D and 2D transfer functions are also embedded into our interface allowing the enhancement and selection of areas and

Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 3

Author Manuscript

features of interest. Our system interacts with 2D or 3D visual presentations of dataset directly and displays the results using the conformal mapping. Our system minimizes global distortions and preserves local angular relationships which, in turn, preserve important shapes and features of the models during the deformation. Our technique can be applied in a wide range of application areas, such as zoom-viewers for maps, texts or videos on smart phones as well as web browsers. It also supports large tree maps as well as tabular and volumetric datasets.


The main contributions of our paper are: (1) A new conformal magnifier, providing a local shape preservation and a good balance between local details in the focus region and the smoothly transiting global context. In this paper, shape preservation means angle preserving, in discrete setting, the angle distortion is minimized; and (2) The application and demonstration of this magnifier within a diverse range of application areas. To the best of our knowledge, no previous work has used conformal mapping theory as a F+C technique for information visualization. In this respect, our conformal magnifier, representing an ideal continuous multi-focus F+C technique that we augmented with several unique features, is a novel non-linear magnification method. The well-defined conformal function is numerically well behaved: in theory, conformal mapping do not have any local angle distortion (in smooth case, or in principle). Namely, at anywhere, there is no angle distortion. Therefore, the deformation induced has free angle distortion. In the discrete setting of triangular meshing structure, the discrete approximation of conformal mapping minimizes the angle distortion. So it can be very efficient to minimize the distortion in the magnified area and be very robust in solving several challenges in the context of visualizing information. Our paper formally proves the following features of our conformal lens: the magnifier design of arbitrary shapes, the local shape preservation with minimized distortion and the smooth transition between focus and context regions. Our paper is structured as follows. First, in Section 2, we present an overview of related work. Then, in Section 3, we present our framework and its methods in detail. Next, in Section 4, we describe the user interface and implementation. Finally, our results are presented in Section 5, and Section 6 ends with conclusions and discusses the future work.

2 Related Work

Author Manuscript

F+C visualization has been addressed in a great number of works, in the context of trees [36] [26], treemaps [10], graphs [25] and tables [31] [40], city and maps [7] and nested networks [41]. The commonly used F+C techniques are lenses and magnifiers, such as fisheye [12] [11], nonlinear magnification transformation [19] [20], detail-in-context [17], distortion [23], multi-scale [7] and so on. Fisheye views, which enlarge the ROI and show other portions of the image with successively less detail, are based on the well-known fisheye lens effect. Fisheye lenses offer an effective navigation and browsing device for various applications [27] [35]. Keahey [18] has conceptualized a treemap as an image to show how to compound zooming with a graphical fisheye lens. InterRing proposed by Yang et al. [47] and Sunburst proposed by Stasko et al. [38] have applied multi-focus fisheye techniques as an important feature for radial space-filling hierarchy visualizations. The fisheye lens displays the data in a continuous manner, without a clear distinction between focus and


Zhao et al.

Page 4

Author Manuscript

context regions. So it does a better job in preserving the spatial relations in the dataset. However, as mentioned, it will create noticeable distortions towards its edges. At the same time, there is no method to formally control the focus region as well as preserve the feature shapes in the context region.

Author Manuscript

More sophisticated lenses and new distortion techniques have therefore been proposed to cope with the shortcomings of basic fisheye lenses. Bier et al. [4] have presented a user interface that enhances the focal interest features and compresses the less interesting regions using a Toolglass and Magic Lenses. Carpendale et al. [5] [6] have proposed several viewdependent distortion patterns to visualize the internal ROI, where more space is assigned for the focal region to highlight the important features. LaMar et al. [21] have presented a fast, natural and intuitive magnification lens with a tessellated border region which estimates linear compression according to the radius of the lenses and the texture information. Pietriga et al. [28] have provided a novel sigma lenses using dimensions called time and translucence to achieve more efficient transitions. The framework allows a unifying presentation space accommodating these new dimensions. Ropinski et al. have [33] proposed a real-time algorithm for rendering volumetric 3D magic lenses with arbitrary convex shapes.


F+C techniques are widely applied for various information visualization structures and displays, including tree, chart, treemap, graph and table. Lamping et al. [22] have proposed a F+C technique based on hyperbolic geometry to visualize large hierarchies. The essence of this scheme is to lay out the hierarchy in a uniform way on a hyperbolic plane and then map this plane onto a circular display region. Plaisant et al. [29] have defined the SpaceTree as a novel tree browser to support exploration in the large node link tree. The algorithm applies dynamic rescaling of branches to best fit the space and includes integrated search and filter functions. F+C techniques can also be used to highlight selected items within a main treemap view to preserve the context as presented in [2]. For seamless focus+context, Shi et al. [35] have proposed a distortion algorithm that increases the size of a node of interest while shrinking its neighbors. Ying et al. [43] have presented a seamless multi-focus and context technique, called Balloon Focus. This method allows the user to smoothly enlarge multiple treemap items serving as the foci, while maintaining a stable treemap layout as the context. Formella et al. [11] have described a system for viewing and browsing planar graphs using a software analog of a fisheye lenses, while Gansner et al. [13] have presented a topological fisheye view for the visualization of large graphs. For large table visualizations, Rao et al. [31] have presented the TableLen as a F+C technique for visualizing and understanding these large tables. The TableLens fuses symbolic and graphical representations into one single coherent view, which can then be controlled by the user. Telear [39] has combined an extended table lens and treemap techniques for the visualization of tabular data, as well as F+C on text, cities and maps. Before embarking on a road trip, travelers often consult Google maps to plan the route. They then print out this map for navigation during the trip. Alternatively, they may carry a GPS device and view and navigate their route this way. In any case, depending on how twisted and expansive the road trip is, the user will see the need for many pages of printouts when planning the trip off-line, or many pan and zoom operations when using the GPS system. He/She never sees the entire trip on one common sheet or display, which makes the planning


Zhao et al.

Page 5

Author Manuscript

of breaks and sightseeing activities difficult. One method to cope with this is the one proposed by Ziegler et al. [48] who have presented an automated system for generating context-preserving route maps. They depict navigation and orientation routes as a path between nodes and edges of a topographic network. This, however, redraws the map only as a graph network, which does not preserve the overall context of the surrounding map constituents, such as other cities, forests, and other information that may become useful on a whim.

Author Manuscript

Qu et al. [30] have proposed a F+C route zoom and information overlay method that is aimed mainly for 3D urban environments. Similarly, Trapp et al. [42] have proposed a 3D generalization lens for interactive F+C visualization and applied it to virtual city models that combines different levels of structural abstraction. They use multiple lenses associated with different abstraction levels simultaneously and further provide the interactive lens modification. For the accurate visualization of 3D volumetric models, Viola et al. [44] have presented an importance driven visualization system to automatically compute the importance of each voxel as a new dimension for rendering to reveal important internal regions from the out layer. Hao et al. have [16] extended the importance-driven mindset for the visualization layouts of large time-series data. Wang et al. [45] have proposed a free-form volumetric lens function to highlight, expose and non-linearly magnify an object in a feature-adaptive and user-configurable fashion. Meanwhile, volume deformation techniques are widely used for the visualization of important features. Wang et al. [46] have presented a method for magnifying features of interest while deforming the context areas without perceivable distortions, using an energy optimization model for large surface models.


All these techniques discussed above have their limitations and are not able to totally solve the F+C problem. Usually, when the focus regions are highlighted, the context may be missed or distorted to some extent; Or the focus may be occluded by the preserved context. Our frame focuses on this problem and produces the optimal visual results: magnify the target in a specific region with free angle distortion and leave the rest of the dataset largely unaffected. Meanwhile, the conformal magnifier contains a transition region so that the user can get a smooth view from the non-magnified outside to the highly magnified inside. Our method is advantageous over the previous approaches, as the transfer function design extremely enhances the expressive power for the user, which can easily satisfy versatile requirements. For the deformation, we apply the conformal mapping as an alternative way to magnify the ROI for the F+C visualization. The application of conformal magnifier perfectly preserves the local shapes and builds a smooth transition field, which lead to a plausible zooming deformation as shown in Figure 2. Meanwhile, the interface enables the user to directly design various models with respect to the different shapes of object in the 2D image. Therefore, the magnifier can effectively cover the entire ROI without moving the magnifier or enlarging non-interest region.


Zhao et al.

Page 6

Author Manuscript

3 Conformal Magnifier The pipeline of our framework is shown in Figure 3. For each input dataset, first, the user selects the ROI through the GUI. Then, a mesh model can be either automatically generated by the default regular magnifier shapes or be manually drawn. After aligning the ROI and the model, the system will automatically display the final magnified result on the screen based on the conformal mapping theory. This section will briefly introduce the conformal mapping theory and algorithm. 3.1 Theoretical Background In this section, we briefly introduce the major concepts in differential geometry and Riemann surface theory, which are necessary to explain the conformal maps. We refer readers to [14, 9] for detailed information.

Author Manuscript

3.1.1 Conformal Structure—Let

be a complex function. The function is

holomorphic, if it satisfies the following Riemann-Cauchy equation

.

Suppose S is a surface embedded in , therefore S has the induced Euclidean metric g. Let Uα ⊂ S be an open set on S, with local parameterization is , such that the metric has local representation

Author Manuscript

then (Uα ϕα) is called an isothermal coordinate chart. We can cover the whole surface by a collection of isothermal coordinate charts. All the isothermal coordinate charts form a conformal structure of the surface. The surface with a conformal structure is a Riemann surface. Suppose S1 and S2 are two Riemann surfaces. Suppose (Uα, ϕα) is a local chart of S1, (Vβ, ψβ) is a local chart of S2. ϕ : S1 → S2 is a conformal map if and only if

is bi-holomorphic. For simplicity, we still use ϕ to denote its local representation. Then a conformal map ϕ satisfies

.

Author Manuscript

3.1.2 Conformal Mapping—Let S be a surface embedded in Euclidean metric g. We say another Riemannian metric scalar function

, such that

The Gaussian curvature induced by

with the induced

is conformal to g, if there is a

. is


Zhao et al.

Page 7

Author Manuscript

where Δg is the Laplacian-Beltrami operator under the original metric g. The above equation is called the Yamabe equation. By solving the Yamabe equation, one can design a conformal metric e2ug by a prescribed curvature

.

Yamabe equation can be solved using Ricci flow method. The Ricci flow deforms the metric g(t) according to the Gaussian curvature K(t) (induced by itself), where t is the time parameter

Author Manuscript

The uniformization theorem for surfaces says that any metric surface admits a conformal metric, which induces constant Gaussian curvature. The constant is one of {−1,0,+1}, determined by the topology of the surface. Such metric is called the uniformization metric. Ricci flow converges to the uniformization metric. Detailed proofs can be found in [15] and [8]. 3.1.3 Discrete Metric and Curvature—In practice, most surfaces are approximated by simplicial complexes, namely triangular meshes. Suppose M is triangle mesh, V,E,F are vertex, edge and face set respectively. We use vi to denote the ith vertex; [vi,vj] the edge from vi to vj; [vi,vj,vk] as the face, where the vertices are sorted counter-clock-wisely.

Author Manuscript

A discrete metric on a mesh M is a function , such that on each triangle [vi,vj,vk], the triangle inequality holds, li + lj > lk. Discrete metric represents a configuration of edge lengths. Suppose M is a mesh with a discrete metric. If all faces are Euclidean, then the mesh is Euclidean background geometry, denoted as . Discrete metric determines the corner angles on each face by co-sine law,

The discrete Gaussian curvature is defined as angle deficient, Suppose M is a mesh with a

Author Manuscript

discrete metric. [vi,vj,vk] is a face in M, represent the corner angle at vi on the face. The discrete Gaussian curvature of vi is defined as

The total Gaussian curvature is controlled by the topology of the mesh,


Zhao et al.

Page 8

Author Manuscript

where Aijk is the area of the face [vi,vj,vk], λ is zero if M is with Euclidean background metric. 3.1.4 Discrete Conformal Deformation—The discrete conformal factor is a function defined on vertices of M, u : . Suppose [vi,vj] is an edge on mesh, ui and uj are the discrete conformal factors on vi and vj. The discrete conformal deformation is Let

.

denote the user defined curvature at vi, the discrete Euclidean Yamabe flow is (1)

Author Manuscript

In discrete conformal deformation, the following differential relation holds,

.

From the above formula, it is clear that, . Therefore, the differential 1-form ΣiKidui is a closed 1-form. The following Yamabe energy is well defined. Let u = (u1,u2, ⋯, un) be the conformal factor vector, where n is the number of vertices, u0 = (0,0, ⋯, 0), then the discrete Euclidean Yamabe energy is (2)

Author Manuscript

The discrete Yamabe flow is the negative gradient flow of the Yamabe energy. Furthermore, we can compute

.

The Hessian matrix (hij) of the discrete Euclidean Yamabe energy can be computed explicitly. Let [vi,vj] be an edge, connecting two faces [vi,vj,vk] and [vj,vi,vl], then the edge weight is defined as

Author Manuscript

also for

where the summation goes through all faces surrounding vi, [vi,vj,vk].


Zhao et al.

Page 9

Author Manuscript

By carefully examining the positive definiteness of the Hessian, the discrete Euclidean Yamabe energy is convex on the space of Σiui = 0, and can be optimized using Newton's method directly. Because the convexity of the energy, there exists a unique global optimum corresponding to the metric with the prescribed curvature

.

Given the mesh M, a conformal factor vector u is admissible, if the deformed metric satisfies triangle inequality on each face. The space of all admissible conformal factors is not convex. In practice, the step length in Newton's method needs to be adjusted. Once triangle inequality doesn't hold on a face, edge swap needs to be performed. 3.2 Algorithm For the conformal mapping, we use the algorithm as follows:

Author Manuscript

The entire computational process for a topological quadrilateral is illustrated in Figure 4. The original mesh model is shown in Figure 4a, with four corner vertices noted as p1,p2,p3,p4. For solving Eq. 1, we demand to set the target curvature for each vertex in the triangle mesh (Figure 4b): Four corner vertices are assigned as π/2, while other vertices as 0. In Figure 4c, the Yamabe flow (i.e., Eq. 1) conformally maps the mesh model onto a planar rectangle, with the corner vertices mapping to the rectangle corners. As a result, we can completely map the input image to the rectangle plane, giving rise to the conformally bijective mapping from the image to the mesh model, as shown in Figure 4d.

Author Manuscript

With the support of the conformal theory and algorithm, we are able to design a magnifier with any arbitrary shape to obtain F+C visualization. As a result of the smooth transition region approach, while the objects inside the center region of the lens are magnified, the objects in the transition region are gradually compressed. Therefore, continuous observation of the objects is achieved with the shape preservation.

4 User Interface and Implementation 4.1 User Interface The interface for the current conformal magnifier system is shown in Figure 5. Interactions for ROI selection, model design and result display, are based on a small number of menu bars, check boxes and pointer gestures using OpenGL, GLUT and FLTK libraries. In particular, we have two modes: selection mode for displaying images and selecting ROI; and design mode for generating mesh mode based on the shape of the selected focus area and zooming scale number. The manipulations under these two modes are supported by control widgets and pointer gestures.

Author Manuscript

4.2 Selection Mode For each input dataset, the user can adjust the 1D or 2D transfer functions to highlight the ROI through the interface. The appropriate use of transfer functions extends the selection choices for the user especially for volumetric datasets. A selection tool is embedded to classify the input into two classes: focus and context. We use two mouse buttons: one for setting (left) and another for adjusting (right). Region selection is performed by pressing the


Zhao et al.

Page 10

Author Manuscript

left button and drawing a stroke on the image. Image is dragged (translation and rotation) using the right button in the selection model. 4.3 Design Mode The design mode supports the generation of mesh model with arbitrary shape and smooth transition between focus and context. It is also a platform for interactive manipulations like adjusting the model shape and setting the appropriate zooming scale inside the focus region.


4.3.1 Basic Operations—The interface supports a number of operations for controlling the shape, size and zooming scale of models in the focal area. In particular, in order to provide a simple interactive manipulation, three canonical manipulation operations are designed as Set, Zoom and Adjust. Each of these three operations can be understood visually as illustrated in Figure 6. Set, by clicking the left button, translates the model to a different position; Zoom increases the model height in the focal area using the slide bar; Adjust, by dragging the right button, modifies the shape of model by adjusting the height (zooming scale) or location for each vertex in the focus area. All the operations are supported by the modeling library of Blender [1]. Both zoom and adjust can change the height of the model: Zoom changes the height of all vertices together; Adjust modifies the height of each vertex to obtain an arbitrary magnification model. Our framework also supports the multiple focal areas design with respect to the multiple ROIs. Each basic operation can be done individually in each focal area or done for all of them together. In a given visualization scenario, the ROIs to be highlighted are not necessary to be assumed to form a simple shape. Therefore, the user should be able to customize the shape of the model to fit the actual boundary of ROIs appropriately. Therefore, the multiple-focal model design opens up an abundant model design to maximally satisfy the user's requirement and save the computational time. 4.3.2 Model Design of Arbitrary Shape—Although the basic operations combining with the setting of multi-focus provide a possible way to generate an arbitrary model, the implements are complicated and time consuming. Therefore, an alternative method is implemented for the arbitrary model design with two steps: point cloud construction and mesh generation from point cloud as shown in Figure 7.

Author Manuscript

Point Cloud Construction: First, the particular boundary curves and a centerline are stroked by the user in the ROI to highlight the focus region as shown in Figure 7a. The zooming slide bar here is used to control the height of all points along the centerline. The system will automatically sample the boundary curve and centerline to discrete sample points based on the user's setting. For each boundary sample point, the Euclidian distance is calculated to find the nearest point at the centerline. The position of each new point will be derived by a calculation based on the pre-defined quadratic function in order to guarantee the smooth transition as shown in Figure 7b. The sampling parameter and the quadratic function can be selected using the slider and check boxes of the interface. High sampling rate generates large points to fit the transition area smoothly but will take a long computational time (using 14.365 sec for 20,000 point cloud).


Zhao et al.

Page 11

Author Manuscript

Mesh Generation from Point Cloud: We provides several functions to generate smooth surface meshes from point cloud based on the Delaunay refinement [34]. First, find the constrained Delaunay triangulation of the input vertices. Next stage is to remove triangles from concavity and hole structures if necessary. The last step is to refine the mesh by inserting additional vertices based on the constraints defined by Ruppert [34]. This algorithm guarantees the accuracy of the surface approximation as shown in Figure 7c. The manifold extraction is also implemented in order to have a regular smooth surface using the ball pivoting method [3], which extracts the manifold and return outwards normals orientation from a given point cloud. In order to obtain the best fitted surface, more points in high curvature features will guarantee the best results. The mesh generation computing speed is mainly effected by the point number, shape and topology of point cloud (e.g. it takes 24.767 sec for our test model with 20,000 points).

Author Manuscript

After loading data, the user can either select the regular shape models or design a specified model (shown in Figure 8) to mark features of interest through the interface. The model design of arbitrary shape enables an extremely rich and natural style of direct manipulation for exploratory data analysis.

Author Manuscript

Meanwhile, in order to optimize the computation, it is not very difficult to embed the mesh design, conformal mapping and image display by the CPU+GPU. We implement our volume rendering and transfer function specification using GPU with the framebuffer objects extension of OpenGL for acceleration. The pixel color and the alpha of an image is controlled by transfer function. The textures need to be computed again each time when some setting is changed, but with real time updating on the application performance. We use the fragment programs for volume rendering: Cast the ray into the volume and composite the color based on the volume data and transfer function; Rendering the result to the framebuffer as output. We use the following configuration: Dell desktop precision PWS670 with Intel Xeon CPU 3.60GHz and 3GB Memory and Nvidia GeForce GTX 285 graphic card. This strategy is good enough for our purposes: interactive operations of the ROI selection, model design and display.

5 Experimental Results We envisage our system having applicability not just to the 2D information structures, maps and satellite images, but for feature exploration in the volumetric datasets. Table 1 shows the designed mesh shapes, vertex/face number, application and computational time of conformal mapping for all the experimental cases used in this paper.

Author Manuscript

5.1 Data Structure Visualization A tree structure is a graphical way of representing the hierarchical nature of a structure. For the access to ancestors/root or to the particularly important nodes, our conformal magnifier makes it easy to find and navigate toward these nodes. In addition, the dynamic nodes distortion of the tree display according to the varying interest levels is a popular application. So our framework provides the scanning function: magnifier can be placed along a path to zoom in the specified region while the remaining regions are evenly distributed back to the non-focus state (shown in Figure 9).


Zhao et al.

Page 12

Author Manuscript

The treemap and tabular are popular methods for information visualization. Treemap will divide the display area into rectangular areas recursively according to the hierarchical structure, which can effectively display the overall hierarchy and some detailed data values. For the table, the most salient feature is the regularity with the information interrelated rows or columns to form a coherent set (e.g. the attributes of an object). So, for the treemap and tabular visualization, we want to preserve the items with rectangular shape and to maintain the same relative positions among the cells after the focus regions are enlarged. Meanwhile, a stable and consistent appearance between the focus and context regions is highly desirable for the user to track individual cell. Conformal magnifier can satisfy these requirements with minimal angular distortion as shown in Figure 10. 5.2 Smartphones/Limited Screen Visualization

Author Manuscript

Smartphones combine the portability and network connectivity of cell-phones with the computing power of PDAs. The limited input modalities of smartphones make it difficult to support both clear interesting region and the context area of large datasets without any extra operations [32]. Our framework improves the magnification functions for the most common styles used by smartphones including texts, charts and logos as shown in Figure 11, Figure 12 and Figure 13.

Author Manuscript

For the traditional google maps, there are two main shortcomings: small roads are not visible because of the constant scale factor; On the other hand, the important and extraneous information can not be distinguished. With current highlighting techniques, small streets the user interest in are not visible and require multiple zooming and panning steps. Our conformal magnifier provides an effective way for map visualization as shown in Figure 14. From these images, the small cities/roads near the ROI are magnified with minimal angle distortion for details view while keeping the entire context regions. We also implement our pipeline for the geological/satellited images as shown in Figure 1 and Figure 15. In the Figure 15, because of the irregular shape in the ROI, an arbitrary model that fits the boundary of ROI is designed. By this arbitrary model, only the park region (ROI) is magnified without any context region. Meanwhile, the smooth transition design between the focus an context region is a nature way for human optical system, which will effectively decrease the visual distortion. 5.3 Volumetric Datasets Visualization

Author Manuscript

Volume visualization helps people achieve insight into volumetric data using interactive graphics and imaging techniques. The dataset from real applications often contains structures of interest, so a good magnifier will be extremely helpful for its visualization and navigation. In order to extend the conformal magnifier as an exploration tool working in 3D space, we replace the traditional 2D screen by our conformal lens. Therefore, the user can position the lens to magnify the 3D ROI within the dataset, regardless of the viewpoint (shown in Figure 16). We have applied our method to different datasets including a time-variable volumetric dataset, to magnify the ROI as shown in Figure 17 and Figure 18. With the mathematical


Zhao et al.

Page 13

Author Manuscript

guarantees of conformal mapping, the volume dataset can be efficiently computed and accurately displayed. 5.4 Multi-Level Magnification Our frame work can be extended to multi-level magnification. The user can interactively reselect ROI and keep on zooming in on the latest magnified result. Figure 20 shows the the first level and second level conformal magnification results in the railway intersection region. The same result can also be generated by only one conformal mapping if we design an arbitrary model with large heights (zooming scales) at center of the focus area (details in Section 4.3). However, from the Table 1, the computing time of conformal mapping is less than 3.8 second while the manually model design is the actual bottleneck in the timing point view. Therefore, the level by level magnification scheme is default in our framework.

Author Manuscript

5.5 Distortion Comparison F+C techniques visualize an entire information structure at once with zooming in on specific items, which supports searching for patterns in the big picture and investigating the details of interest without losing the context. In order to address this purpose, F+C methods usually impose spatial distortion, in such a way that the focus region is displayed with accurate magnification and the context region with distortion - but it must be still kept visible. In this paper, we implement several practical examples of F+C techniques including the bifocal display [37], the perspective wall [24], the fisheye technique [12] and the poly-focus lens as the distortion comparison. The 2D distortion fields are shown in Figure 19.

Author Manuscript

To prove that our framework can provide a better understanding of the magnification effect in a real scenario, we carry out the evaluation with GUI users. Every subject rates each one of the deformed results according to the user's personal preference - from bad to excellent (1–4). The statistics result using each subject is shown in the Table 2. As shown in the table, the traditional zooming effect does not have a very good acceptance, as the cropping effects the connection between the context and the focus region, making the image unsuitable for complete comprehension. Similar results are obtained for the bifocal and perspective wall methods, due to the unusual distortion produced by them. The fisheye, poly-focus and conformal magnifier can preserve the entire information of the image with gradually decreasing degree of shape distortion. The numbers support that our conformal magnifier is accepted by most users.

6 Conclusions and Future Work Author Manuscript

In this paper, we present a simple but powerful conformal magnifier as a F+C technique to overcome the limitations of screen space. First, an interface is designed to provide various models based on the shape of the interest object in the image. Then, the user selects zooming scale and sets deformation constraints through this interface. Next, conformal mapping method is applied to automatically magnify the ROI with minimal angle distortion while keeping the entire context region. The deformation can work directly on the various datasets including texts, maps, structures and 3D volume. With the support of experiment results of


Zhao et al.

Page 14

Author Manuscript

various cases, we prove that our conformal magnifier is a helpful F+C technique and has a great potential in the volume graphics and visualization applications. However, because the transition region of the mesh model is not smooth enough, part of image is too compressed to visualize any detail. Some specified filters are further demanded to smooth the transition region of mesh models to obtain high quality result. On the other side, as the graphics hardware continues to improve, in the future we expect to further accelerate the conformal calculation by GPU. It may be also possible to incorporate better quality rendering method for high-performance and resolution.

REFERENCES

Author Manuscript Author Manuscript Author Manuscript

[1]. Homepage of blender. www.blender.org [2]. Treemap, the human-computer interaction lab of University of Maryland. www.cs.umd.edu/hcil/ treemap [3]. Bernardini F, Mittleman J, Rushmeier H, Silva C, Taubin G. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics. 1999; 5:349–359. [4]. Bier EA, Stone MC, Pier K, Buxton W, Derose TD. Toolglass and magic lenses: The see-through interface. Computer Graphics. 1993:73–80. [5]. Carpendale MST, Carpendale T, Cowperthwaite DJ, Fracchia FD. Distortion viewing techniques for 3-dimensional data. IEEE Symposium on Information Visualization. 1996:46–53. [6]. Carpendale MST, Cowperthwaite DJ, Fracchia FD. Extending distortion viewing from 2D to 3D. IEEE Comput. Graph. Appl. 1997; 17(4):42–51. [7]. Carpendale MST, Sheelagh M, Carpendale T, Cowperthwaite DJ, Fracchia FD. Multi-scale viewing. SIGGRAPH. 1996:149–152. [8]. Chow B. The ricci flow on the 2-sphere. J. Differential Geom. 1991; 33(2):325–334. [9]. Farkas, HM.; Kra, I. Riemann Surfaces. Springer; 2004. [10]. Fekete JD, Plaisant C. Interactive information visualization of a million items. IEEE Symposium on Information Visualization. 2002:117–124. [11]. Formella A, Keller J. Generalized fisheye views of graphs. Proceedings Graph Drawing, Lecture Notes in Computer Science, LNCS 1027. 1995:242–253. [12]. Furnas, G. Generalized fisheye views. Human Factors in Computing Systems, CHI Conference Proceedings; 1986. p. 16-23. [13]. Gansner E, Koren Y, North S. Topological fisheye views for visualizing large graphs. IEEE Symposium on Information Visualization. 2004:175–182. [14]. Guggenheimer, HW. Differential Geometry. Dover Publications; 1977. [15]. Hamilton RS. Three manifolds with positive ricci curvature. Journal of Differential Geometry. 1982; 17:255–306. [16]. Hao MC, Dayal U, Keim DA, Schreck T. Importance-driven visualization layouts for large time series data. IEEE Symposium on Information Visualization. 2005:27–34. [17]. Keahey TA. The generalized detail-in-context problem. IEEE Symposium on Information Visualization. 1998:44–51. [18]. Keahey TA. Getting along: Composition of visualization paradigms. IEEE Symposium on Information Visualization. 2001:37–40. [19]. Keahey TA, Robertson EL. Techniques for non-linear magnification transformations. IEEE Symposium on Information Visualization. 1996:38–45. [20]. Keahey TA, Robertson EL. Nonlinear magnification fields. IEEE Symposium on Information Visualization. 1997:51–58. [21]. LaMar, E.; Hamann, B.; Joy, KI. A magnification lens for interactive volume visualization. Pacific Conference on Computer Graphics and Applications; 2001. p. 223-232.


Zhao et al.

Page 15

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

[22]. Lamping, J.; Rao, R.; Pirolli, P. A focus+context technique based on hyperbolic geometry for visualizing large hierarchies. Conference on Human Factors in Computing Systems (CHI); 1995. p. 401-408. [23]. Leung YK, Apperley MD. A review and taxonomy of distortion-oriented presentation techniques. ACM Transactions on Computer-Human Interaction. 1994; 1(2):126–160. [24]. Mackinlay, JD.; Robertson, GG.; Card, SK. The perspective wall: Detail and context smoothly integrated. Conference on Human Factors in Computing Systems (CHI); 1991. p. 173-179. [25]. Munzner T. H3: laying out large directed graphs in 3D hyperbolic space. IEEE Symposium on Information Visualization. 1997:2–8. [26]. Munzner T, Guimbretière F, Tasiran S, Zhang L, Zhou Y. Treejuxtaposer: Scalable tree comparison using focus+context with guaranteed visibility. ACM Transactions on Graphics. 2003; 22:453–462. [27]. Nekrasovski, D.; Bodnar, A.; McGrenere, J.; Guimbretière, F.; Munzner, T. An evaluation of pan & zoom and rubber sheet navigation with and without an overview. CHI Conference on Human Factors in Computing Systems; 2006. p. 11-20. [28]. Pietriga, E.; Appert, C. Sigma lenses: focus-context transitions combining space, time and translucence. Conference on Human Factors in Computing Systems (CHI); 2008. p. 1343-1352. [29]. Plaisant C, Grosjean J, Bederson BB. Spacetree: Supporting exploration in large node link tree, design evolution and empirical evaluation. IEEE Symposium on Information Visualization. 2002:57–64. [30]. Qu H, Wang H, Cui W, Wu Y, Chan M-Y. Focus+context route zooming and information overlay in 3D urban environments. IEEE Transactions on Visualization and Computer Graphics. 2009; 15:1547–1554. [PubMed: 19834232] [31]. Rao, R.; Card, SK. The table lens: Merging graphical and symbolic representations in an interactive focus+context visualization for tabular information. Conference on Human Factors in Computing Systems (CHI); 1994. p. 222-228. [32]. Robbins, DC.; Cutrell, E.; Sarin, R.; Horvitz, E. Zonezoom: map navigation for smartphones with recursive view segmentation. Working Conference on Advanced Visual Interfaces (AVI); 2004. p. 231-234. [33]. Ropinski T, Hinrichs K. Real-time rendering of 3D magic lenses having arbitrary convex shapes. Journal of the International Winter School of Computer Graphics. 2004:379–386. [34]. Ruppert J. A delaunay refinement algorithm for quality 2-dimensional mesh generation. IEEE Transactions on Visualization and Computer Graphics. 1995; 18(3):548–585. [35]. Shi K, Irani P, Li B. An evaluation of content browsing techniques for hierarchical space-filling visualizations. IEEE Symposium on Information Visualization. 2005:11–18. [36]. Slack J, Munzner T. Composite rectilinear deformation for stretch and squish navigation. IEEE Transactions on Visualization and Computer Graphics. 2006; 12:901–908. [PubMed: 17080815] [37]. Spence R, Apperley M. Database navigation: An office environment for the professional. Behavior and Information Technology. 1982; 1(1):43–45. [38]. Stasko J, Zhang E. Focus+context display and navigation techniques for enhancing radial, spacefilling hierarchy visualizations. IEEE Symposium on Information Visualization. 2000:57–64. [39]. Telea A. Combining extended table lens and treemap techniques for visualizing tabular data. Eurographics/ IEEE-VGTC Symposium on Visualization (EuroVis). 2006 [40]. Tenev T, Rao R. Managing multiple focal levels in table lens. IEEE Symposium on Information Visualization. 1997:59–66. [41]. Toyoda, M.; Shibayama, E. Hyper mochi sheet: a predictive focusing interface for navigating and editing nested networks through a multi-focus distortion-oriented view. Conference on Human Factors in Computing Systems; 1999. p. 504-511. [42]. Trapp, M.; Glander, T.; Buchholz, H.; Dollner, J. 3D generalization lenses for interactive focus +context visualization of virtual city models. International Conference on Information Visualisation; 2008. p. 356-361. [43]. Tu Y, Shen H-W. Balloon focus: a seamless multi-focus+context method for treemaps. IEEE Transactions on Visualization and Computer Graphics. 2008; 14:1157–1164. [PubMed: 18988959] Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 16

Author Manuscript

[44]. Viola I, Kanitsar A, Groller ME. Importance-driven volume rendering. IEEE Visualization. 2004:139–145. [45]. Wang L, Zhao Y, Mueller K, Kaufman A. The magic volume lens: An interactive focus+context technique for volume rendering. IEEE Visualization. 2005:367–374. [46]. Wang Y-S, Lee T-Y, Tai C-L. Focus+context visualization with distortion minimization. IEEE Transactions on Visualization and Computer Graphics. 2008; 14:1731–1738. [PubMed: 18989032] [47]. Yang J, Ward MO, Rundensteiner EA, Patro A. Interring: a visual interface for navigating and manipulating hierarchies. IEEE Symposium on Information Visualization. 2003; 2(1):16–30. [48]. Ziegler H, Keim DA. Copernicus: Context-preserving engine for route navigation with interactive user-modifiable scaling. Comput. Graph. Forum. 2008; 27(3):927–934.

Author Manuscript Author Manuscript Author Manuscript Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 17

Author Manuscript Fig. 1.

Author Manuscript

The conformal magnification of the western earth using our conformal magnifier. (a) Original image. (b) The conformal magnified result to highlight the North America.

Author Manuscript Author Manuscript Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 18


Author Manuscript

Basic virtue of the conformal mapping. (a) A checker board image with the letter “T”. (b) Conformal mapping preserves angles and the orthogonal features after the magnification. (c) The angle distortion of letter “T” will not be identifiable visually.


Zhao et al.

Page 19

Author Manuscript

Fig. 3.

A schematic diagram.


Zhao et al.

Page 20


Fig. 4.

Conformal mappings for a topological quadrilateral surface: (a) The original surface. p1, p2, p3, p4 are four corners. (b) The triangular mesh structure. (c) The image of the conformal map, which is a rectangle. (d) The checker board texture mapping through the rectangular conformal map, where the right angles are preserved.


Zhao et al.

Page 21


Fig. 5.

An overview of our interface. The main window shows the conformal mapping result of Vis logo while the left display window shows the hemisphere mesh model we use for conformal mapping.


Zhao et al.

Page 22

Author Manuscript Author Manuscript Fig. 6.

Author Manuscript

Three basic operations for mesh design. (a) Original designed mesh model. (b) The set operation changes the center position of the original model; (c) The zoom operation changes the height of the original model; (d) The adjust operation re-sets the height or position of vertices of the original model.

Author Manuscript Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 23


Author Manuscript

Two steps to design a model with an arbitrary shape. (a) The boundary and centerline drawn by the user on the input image. (b) The point cloud generation with respect to the boundary and centerline drawn in (a). The cyan line is a quadratic curve to generate the 3D sampling points between a boundary point and its nearest point at the centerline. (c) The generated mesh based on the point cloud in (b). The black dashed frame shows the magnified details of the point cloud (b) and the generated mesh (c).


Zhao et al.

Page 24


Author Manuscript

Mesh models. (a) Regular shapes (hemisphere, Gaussian and rectangle). (b) Arbitrary shapes (two roof, any curve and Gaussian plus rectangle).


Zhao et al.

Page 25


The conformal magnification results of force-directed tree. (a) Original image. (b) The conformal magnification result of root. The conformal magnification results of right nodes (c) and left nodes (d).


Zhao et al.

Page 26


The conformal magnification result of Treemap. (a) Original image. (b) The conformal magnification result to zoom in the center region while preserving the rectangular edges.


Zhao et al.

Page 27


Author Manuscript

The conformal magnification results of text. (a) Original image of taxonomic tree. The conformal magnification results of top part (b) and bottom part (c) of (a).


Zhao et al.

Page 28

Author Manuscript

Fig. 12.

The conformal magnification results of pie chart. (a) Original image. (b) (c) (d) The conformal magnification results from different focus points to highlight each individual chart.


Zhao et al.

Page 29


The conformal magnification result (b) of Vis10 logo (a).


Zhao et al.

Page 30


Author Manuscript

The conformal magnification results of maps. (a) Original USA map. (b) The conformal magnification result to highlight the neighboring cities around Washington D.C in (a). (c) Original road map. (d) The conformal magnification result to highlight the neighboring blocks along the main highway.

Author Manuscript Visualization (Los Alamitos Calif). Author manuscript; available in PMC 2015 August 14.

Zhao et al.

Page 31


Author Manuscript

The conformal magnification result of geological image. (a) Original image. (b) The conformal magnification result to highlight the park area. The shape of park boundary curve (green line) and the directions of roads (red lines) are perfectly preserved without any visual distortion because of the conformal mapping.


Zhao et al.

Page 32


Raycasting for volumetric (a) common plane and (b) designed plane. During the ray casting, the accumulated value will be stored and shown on the designed plane (green curve marked as P') instead of the common 2D plane.


Zhao et al.

Page 33


Author Manuscript

The conformal magnification result of feature based volumetric dataset. (a) Original image with features of interest colored as red (by the appropriate setting of transfer function). (b) The conformal magnification result to show the details of the internal features of interest.


Zhao et al.

Page 34


The conformal magnification results of the simulated smoke data. (a) and (c) Original volume rendering results. (b) and (d) The conformal magnification results with different focus positions to reveal the local details during the simulation.


Zhao et al.

Page 35

Author Manuscript

Fig. 19.

2D distortion fields of various lenses: (a) Bifocal, (b) Perspective wall, (c) Fisheye, (d) Polyfocal and (e) Conformal.


Zhao et al.

Page 36


Multi-level conformal magnification. (a) Illustrative image of the multi-level conformal magnification. (b) Original metrorail image. The first-level (c) and second-level (d) magnification results.


Author Manuscript

Author Manuscript

Author Manuscript

Park

Sphere

Smoke

Geo

Vol

Media

USA

G Map

FD

Tree

Com

Taxon

Text

Road

Vis10

Logo

G Map

Earth

Sat

Treemap

Name

Case

A Two S S

1283 643

G

5122 5122

E

R

644 * 1454

792 * 584

S half S

Rf

1502

2562

S

768 * 2048

Model

Size 10242

3446

2932

2639

2192

3691

3125

1723

3446

3257

3446

No.V

6826

5739

5328

4830

7063

6018

3413

6826

6451

6826

No.f

3.11

2.78

2.65

2.19

3.76

3.06

1.82

3.61

2.94

3.24

Time(s)

Statistics of various cases applied as experimental results. For the column of model, S: hemisphere, A: arbitrary model, R: rectangle, E: ellipse, G: Gaussian and Rf: roof.

Author Manuscript

Table 1 Zhao et al. Page 37


Zhao et al.

Page 38

Table 2

Author Manuscript

Statistics of various magnifiers (Unit: percentage). Magnifier/Score

1

2

3

4

Zooming

63.49

Bifocal

52.38

36.51

0

0

39.68

7.93

0

Perspective Wall

50.79

46.03

3.17

0

Fisheye

0

60.32

28.57

11.11

Poly-focus

0

26.98

50.79

22.22

Conformal

0

0

20.63

79.36


Stent distortion after sutureless aortic valve implantation: a new complication seen with a novel surgical technique.

Stereotactic conformal radiotherapy for pituitary adenomas: technique and preliminary experience.

Commentary on Bolster et al. Cubital tunnel syndrome: a comparison of an endoscopic technique with a minimal invasive open technique.

[Endonasal paranasal sinus surgery with the binocular magnifier].

Dose escalation for stage C (T3) prostate cancer: minimal rectal toxicity observed using conformal therapy.

Transanal minimal invasive surgery with the Endorec(TM) trocar: a low cost but effective technique.

Finer Needles Reduce Pain Associated With Injection of Local Anesthetic Using a Minimal Insertion Injection Technique.

Counterpoint. A 3D-conformal technique is better than IMRT or VMAT for lung SBRT.

The principle and physical models of novel jetting dispenser with giant magnetostrictive and a magnifier.

Maximizing setup accuracy using portal images as applied to a conformal boost technique for prostatic cancer.

GenInfoGuard--a robust and distortion-free watermarking technique for genetic data.

Minimal invasive and computer-assisted total knee replacement compared with the minimal invasive technique: a prospective, randomized trial with short-term outcomes.

Minimal Modification to the Trerotola Backbleeding Technique Increases Its Effectiveness.

Extracorporeal instrument knotting technique for minimal access thoracic surgery.

Limitations of minimal incision technique for tibiotalocalcaneal arthrodesis.

Vulnérologie: un néologisme pour magnifier le concept «plaies et cicatrisation».

A convenient and less invasive technique of labial minor salivary gland biopsy using a minimal incision with a needle tip.

Conformal therapy.

Conformal Infinity.

Technique for comprehensive head and neck irradiation using 3-dimensional conformal proton therapy.

Surgical closure of persistent arterial duct with minimal invasive anterior thoracotomy: an alternative technique.

Effective and Efficient Stand Magnifier Use in Visually Impaired Children.

The minimal-access kidney transplantation technique in living-donor transplantation: results from a retrospective analysis.

Arthroscopic technique for treatment of combined pathology associated with femoroacetabular impingement syndrome using traction sutures and a minimal capsulotomy.