This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
1
Experimental Study on the Perception Characteristics of Haptic Texture by Multidimensional Scaling Juan Wu, Member, IEEE, Na Li, Wei Liu, Guangming Song, Member, IEEE，and Jun Zhang，Member, IEEE Abstract—Recent works regarding real texture perception demonstrate that physical factors such as stiffness and spatial period play a fundamental role in texture perception. This research used a multidimensional scaling (MDS) analysis to further characterize and quantify the effects of the simulation parameters on haptic texture rendering and perception. In a pilot experiment, twelve haptic texture samples were generated by using a 3-degrees-of-freedom (3-DOF) force-feedback device with varying spatial period, height and stiffness coefficient parameter values. The subjects’ perceptions of the virtual textures indicate that roughness, denseness, flatness and hardness are distinguishing characteristics of texture. In the main experiment, nineteen participants rated the dissimilarities of the textures and estimated the magnitudes of their characteristics. The MDS method was used to recover the underlying perceptual space and reveal the significance of the space from the recorded data. The physical parameters and their combinations have significant effects on the perceptual characteristics. A regression model was used to quantitatively analyze the parameters and their effects on the perceptual characteristics. This paper is to illustrate that haptic texture perception based on force feedback can be modeled in two- or three-dimensional space and provide suggestions on improving perception-based haptic texture rendering. Index Terms—Virtual texture perception, multidimensional scaling, texture perceptual space
————————————————————
1 INTRODUCTION IRTUAL haptic texture rendering transmits the con-
V
tact perception of surface roughness to operators through haptic display devices. With the development of haptic texture rendering technology, virtual haptic rendering has been widely used in many areas, such as biomedical engineering [1] [2], military simulations [3] [4], art galleries [5], and teleoperation. Particularly essential applications include enhancement of the sense of reality of the blind and human-computer interaction (HCI). Prior to simulating real textures, it is essential to research the inherent mechanisms of real tactile texture perception. It is necessary to have a better understanding of the generation of texture sensation and the physical factors that influence the perception of texture. Previous research studies have contributed much to real texture perception, and its characteristics have been under research for years. Real textures can be classified into artificial textures and natural textures. The artificial textures, such as paper and silk, are created and designed by human beings while the natural textures, such as the textures of water, stone, and wood, come into being naturally. Although most all of the textures employed in ————————————————
The authors are with the Robotic Sensor and Control Laboratory, School of Instrument Science and Engineering of Southeast University, No.2, Sipailou, Nanjing, China. E-mail: 220132608@ seu.edu.cn.
xxxx-xxxx/0x/$xx.00 © 200x IEEE
previous studies are artificial textures and can be experienced by direct touching, these studies vary in terms of their mathematical approaches, methods of psychological experiment as well as the stimuli. These distinct factors resulted in different experimental findings. To explore the characteristics of real texture perception, Yoshida researched the major dimensions of touch using 25 samples with various textures [6] and discovered that the most important dimensions were heaviness, coldness, wetness, smoothness and hardness. According to Hollins et al. who used 17 tactile stimuli to research the subjective dimensionality of tactile surface texture perception [7], roughness-smoothness and hardness-softness were robust orthogonal dimensions. According to Picard et al. who researched the perceptual dimensions of everyday tactile textures [8], the perceptual dimensions were softness/harshness, thinness/thickness, relief and hardness. And according to Ballesteros et al. who analyzed the perceptual space of 20 materials, there was a two-dimensional (2D) space in which the first dimension was roughness/smoothness and the second dimension was slipperiness/stickiness and hardness/softness [9]. They also proposed a three-dimensional (3D) space in which the three dimensions were roughness/smoothness, slipperiness/stickiness, and hardness/softness [10]. Summers et al. constructed a perceptual space of 10 types of paper, and the first and second dimensions were predicted to be the roughness and stiffness, respectively [11]. Guest et al. analyzed the tactile dimensions of five types of fabrics [12] and found Published by the IEEE Computer Society
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 2
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
that the extracted dimensions included the roughness/smoothness, moisture/drought, and hardness/softness. And similar dimensions have been discovered in different body sites. The perception of real texture can also be affected by its contact mode, whether touched with a finger or with a probe. Texture perception with a probe can be influenced by roughness [13], hardness [14], and stickiness [15]. Yoshioka compared the texture perception from direct touch with that from indirect touch via a probe held in one hand [16]. He concluded that there were 3D spaces for both types of touch and identified the dimensions as roughness, hardness, and stickiness. He also discovered that the texture perceptions were similar, although not identical, in the two scanning modes. In addition, coldness/warmness cannot be conveyed by probes. Okamoto et al. reviewed the existing research regarding the tactile dimensionality of the physical properties of different materials [17]. Based on the analyzed studies, the authors concluded that the following three distinguishing characteristics composed the haptic texture perceptual space, regardless of whether the perception was obtained via a device or by direct touch: roughness/smoothness, hardness/softness, and stickiness/slipperiness. A virtual texture is rendered differently from a real one because of the performance limits of hardware and the exploration modality. Because of the point contact and kinetic characteristics of haptic textures, many researches regarding haptic textures have been performed with force feedback devices. However, much of the existing literature about virtual textures focuses on algorithms and hardware rather than perceptual characteristics or relationships between perceptual characteristics and the properties of the simulated textures. Minsky et al. [18] extended control algorithms and spring forces based on local gradients to create textured surfaces. In later research [19], they simulated haptic surface textures by using lateral forces and examined the textural perception of roughness for varied force characteristics and spatial geometries. Their results suggest that roughness is rated along a single-axis physical scale that depends on the force amplitude [19]. Ahmaniemi et al. [20] designed a haptic texture display system by using a single vibrotactile actuator and motion sensor. They concluded that the ridge length and spatial denseness influenced the perceived roughness and flatness. Unger et al. [21] performed virtual texture perception experiments with highfidelity magnetic levitation and developed a psychophysical function for roughness perception. They discovered an inverted “U”-shaped function between the roughness and the inter-element spacing that was nearly identical to that of real surfaces. Colwell et al. [22] used a magnitude estimation method to research the roughness of ten simulated textures, and they found the crucial relationship between the roughness perception and the simulated physical parameters. These previous studies of virtual texture forces and their perception are fruitful. However, the results of these studies are not comparable because of their diversity in
terms of facilities, modeling approaches and data analysis methods. Most of the studies concentrate on one specific perceptual characteristic of virtual texture, especially the roughness. They revealed that the perception of roughness was affected by the physical parameters. For example, Ahmaniemi researched the effects of envelope ridge lengths, spatial densities, and regularity on the perceptual characteristics of a virtual texture. The haptic texture was generated by a single vibrotactile actuator and motion sensors instead of force feedback devices. However, the relationships between the actual perceptual characteristics other than roughness and the simulated parameters of force-based virtual textures have not aroused the researchers’ interest yet. This paper intends to analyze the relationship between the perceptual characteristics and the simulated parameters based on general force feedback devices, which is necessary to uncover the inherent connection between the input and output of haptic texture rendering systems and to make haptic rendering systems more realistic. To clarify the perceptual characteristics of virtual textures generated by force feedback, the research team applied a general 3-degrees-of-freedom (3-DOF) force feedback device to create tactile force stimuli and try to reveal the virtual haptic texture perceptual space with a basic and simple haptic texture model. A standardized periodic texture signal was used to simulate the tactile sense, similar to regular textures. The haptic texture was simulated when a user manipulated the force feedback device so as to help it move over the surface of the virtual texture. The multidimensional scaling (MDS) method was used to analyze the experiments. From the perspective of psychophysics, the researchers further explored how objective parameters affected human texture perception and how people distinguished different haptic textures. The relationship between the perceptual characteristics of the virtual texture and the objective parameters has also been studied. As a result, some valuable suggestions for further improving the haptic texture rendering based on force feedback will be provided.
2 SYSTEM AND METHODS 2.1 MDS Analysis MDS is a method to measure the similarity (or dissimilarity) among pairs of objects as distances between points in a low-dimension multidimensional space [23]. The output of the MDS method is a configuration of objects embedded in a multidimensional space. The MDS technique has been widely applied in tactile and haptic perception studies [24] [25] [26] [27]. The data analysts can observe the data and display according to the graphical display of the correlations provided by MDS [28]. MDS analysis can provide information concerning the psychological representation of objects, including the dimensionality of their perceptual space, interpretation of their dimensions, and configuration of the objects in their perceptual space. The K-stress and RSQ are applied to determine the fitness between the dimensionality and the observed data. The K-stress is a normalized difference between the fitted dis-
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics AUTHOR ET AL.: TITLE
3
Fig. 2. Schematic of the haptic interaction algorithm.
Fig. 1. (a) A real texture perception system based on point contact.
Fig. 3. Schematic of the texture height used in the experiments.
to control the virtual ball at a specified velocity. The speed was kept constant so that the results could exclude the effects of varying speed [30] [31].
2.3 Texture Force Rendering Algorithm
Fig. 1. (b) The virtual texture perception system used in our experiments, which simulates a point contact with a real texture surface.
tances and the observed proximities. Stress values of less than 0.2 and a sudden decrease in the stress values on the plot indicate that the output fits the similarity data well [29]. In this study, human perceptual dissimilarity ratings of a set of haptic textures were used as the input data. Stress values were calculated for one- through- fivedimensional solutions. According to the stress values and the trend of the plot, output configurations were obtained; these configurations could be interpreted as maps of the haptic textures in a psychological space.
2.2 Experimental Design The system applied in our experiments is as shown in Fig.1, of which, Fig.1. (a) is a real texture perception system used in [16], while Fig.1. (b) is our virtual texture perception system. In contrast with the real texture perception system, the virtual system uses the relatively inexpensive, portable, safe, and reliable 3-DOF Omni force feedback device manufactured by SensAble Corp. (located in the city of Wilmington, Massachusetts) for texture display. The force feedback device has a position resolution of 0.055 mm and can provide a maximum force of 3.3 N. In the real texture perception system, an operator placed a probe on a real texture. In the experiment, a user held the Omni probe to make the haptic interface point (HIP) move back and forth on the textured surface. The appropriate force was generated and pushed back to the user to simulate a collision through the probe. In the experiments, subjects followed the target ball on the screen
A texture force can be modeled based on physical measurement, geometrical constraints and a physical model or image processing [32][33][34]. Previous experimental studies suggest that the main characteristics of the real textures are roughness/smoothness, hardness/softness, stickiness/slipperiness, and flatness/bumpiness. The main factors that influence those characteristics include the spatial period, height and stiffness [35][21][36][37]. Therefore, to synthesize the haptic texture, the researchers chose the waveform, spatial period, height and stiffness coefficient as the parameters. As the focus of this paper is the physical factors of texture surface, the texture force model is simplified and only related to the physical properties. The virtual surface is idealized as a regular square wave. The HIP is inside the virtual object when the device comes in contact with the object, as shown in Fig. 2. The center of the circle represents the HIP. The force is perpendicular to the associated surface, and the magnitude is a function of the vertical distance from the surface to the HIP. The force is calculated and actuated according to Hooke’s law, a simple linear force model: F=ky (1) where k is the stiffness coefficient and y is the vertical distance. The texture is created by the 3-DOF Omni force feedback device. To simplify the analysis and eliminate the other factors which would influence the experimental results, the research used an ideally textured surface that was similar to that used in [25]. The texture force model was an idealized square wave signal that was perpendicular to the virtual texture surface. The researchers established a 3D space of the virtual texture with the Cartesian space, which was defined by the Oxyz coordinates. The texture height hZ was along the z axis. It was perpendicular to the
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 4
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
texture surface and changed periodically with changes in the x-axis coordinate, as shown in Fig. 3. The y-axis is not shown in the figure because the height do not change in the y direction. The height of the texture can be expressed as
ℎ =
ℎ, sin(2 0, sin(2
/ /
) > 0 )≤0
(2)
where is the horizontal axis (x-axis) coordinate of the texture surface and ℎ is the height of the square wave, denotes the spatial period. The which is constant, and duty cycle ratio is 1:1. Except the tangential friction, the feedback function of the texture force can be expressed as
=
(ℎ − ), ≤ℎ 0, others
(3)
is the amplitude of the texture force, is the Here, position of the HIP on the z-axis, and k is the stiffness coefficient. When the handle moves into the texture plane, ≤ ℎ , there is an output of force feedback. where The texture parameters of the texture force model in our research were set as follows. According to our previous study, spatial periods in the range of 0.5-15 mm have a significant effect on the perception of the texture force [38][39]; thus, the three spatial periods d0 used in the experiments were within this range: 0.5 mm, 6.9 mm and 15 mm. Then, according to the work of Lawrence et al. and Liu [37] [40], along with consideration of the range of stiffness of the device, the height of the haptic texture h was set to 5 and 25 mm, and the stiffness coefficient was set to 0.1 and 1.0 N/mm. Using all possible combinations of these parameters yields 12 different haptic textures, as listed in Table 1.
3 TEXTURE PERCEPTION EXPERIMENTS 3.1 Pilot Experiment Based on previous studies, the texture spatial period, texture height and stiffness coefficient were chosen as the objective stimulus parameters, and 12 virtual textures were synthesized. The purpose of the pilot experiment was to clarify how subjects would describe the haptic TABLE 1 12 VIRTUAL TEXTURES USED IN THE EXPERIMENT
texture when no guidelines or constraints were provided by the experimenters, i.e., the researchers expected to confirm whether the haptic texture reflected the characteristics of the real texture. The goal of this experiment was to provide some insights into the identity of different surface textures and to help discover the differences between the perception of real and virtual textures when the same objective parameters were applied. The experiment would also yield a list of characteristics to describe the feeling of the textured surface.
3.1.1 Participants Eight subjects participated in this experiment. Their ages ranged from 20 to 30, all of whom were right-handed and reported to have no known cutaneous or kinesthetic problems.
3.1.2 Procedures
The subjects sat at the experiment table with a computer screen and the Omni force feedback device placed on it (see Fig. 1. (b)). In each trial, the subject moved the probe of the Omni across a textured surface and felt the virtual texture. Then, he or she described what he or she felt about those textures after or during the experiment. Twelve virtual textures were presented in a random order.
3.1.3 Results
The subjects provided the following descriptions. Participant 1: smooth, concave-convex, rough, soft, hard. Participant 2: smooth, wide. Participant 3: frequency-dependent, magnitude of the force, rough, smooth, height. Participant 4: amplitude, frequency, duty cycle. Participant 5: magnitude of the force, rough, concaveconvex. Participant 6: rough, smooth, level of fluctuation. Participant 7: periodicity, concave-convex. Participant 8: soft, hard. The researchers classified the words provided by the subjects into roughness (smooth and rough), denseness (wide, frequency-dependent, frequency, duty cycle, and periodicity), flatness (concave-convex, height, amplitude, and level of fluctuation) and hardness (soft, hard, and magnitude of the force). Denseness is a characteristic that is related to the spatial period and can be expressed as coarse or fine. Roughness (smooth/rough), hardness (hard/soft) and flatness (flat/bumpy) are characteristics that most people are familiar with. Comparing those characteristics with real texture perception, we observed that the haptic textures created by combinations of those objective parameters had almost the same characteristics as real textures. Then, in the main experiment, the subjects were asked to evaluate those characteristics for all 12 textures. Through the MDS method, the characteristics used by people to distinguish different virtual textures in different perceptual spaces were studied.
3.2 Formal Perception Experiment The formal perception experiment was conducted using
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics AUTHOR ET AL.: TITLE
the 12 virtual textures. The purpose of the experiment was to study the correlation between the input and output of the system and to determine whether these parameters and perceptual characteristics provide a useful method for interpreting the model of the haptic texture perceptual space. Adjustable simulation parameters, including the spatial period, height and stiffness coefficient, were recorded in the system. The perceptual characteristics including roughness, denseness, flatness and hardness were observed. In the experiment, textures with different combinations of the adjustable simulation parameters were presented to the subjects. A scale from 0 to 10 was adopted to assess the perceptual differences of those textures and the magnitude of each characteristic. The experiment was conducted in two sessions. In the first step, a dissimilarity matrix that distinguished between the perceived dissimilarities of the two stimulus surfaces was observed, where the MDS method was used to establish a satisfactory perceptual space on the basis of this dissimilarity matrix. And in the second step, subjects were asked to provide a quantitative description for each characteristic on a scale from 0 to 10, where three-way repeated measure ANOVA was used to analyze the correlation between the stimulation parameters and perceptual characteristics. Because the objective parameters and perceptual characteristics were the inputs and outputs of the system respectively, the researchers studied the composition of the dimensions from those two aspects. Then, the quantitative relationship between the input and output and the orthogonality of the subjective perceptual characteristics were analyzed. These two steps enabled the researchers to identify the basis by which people distinguish between different virtual textures.
3.2.1 Participants Nineteen subjects who ranged from 20 to 30 years old participated in this experiment. None of them had participated in the pilot experiment. And all were right-handed and reported to have no known cutaneous or kinesthetic problems.
3.2.2 Procedures
The purpose of the main experiment was to obtain data that indicated the perceived dissimilarity of the textures and evaluate the characteristics of each texture. To exclude the influences of the exploratory speed, a constant speed was applied for the perception of different subjects and different textures. According to the previous studies and the habitual exploratory speed of different subjects, 50 mm/s was selected for the experiment. All of the subjects needed some practice to maintain the constant speed by following the target ball on the screen. In the experi-
5
ment process, the subjects were allowed to explore the stimulus in the horizontal direction by scanning back and forth until they were satisfied. Subjects were instructed to report their initial impression of the virtual textures rather than trying to determine the “correct” answer.
Step 1: The 12 textures were compared with each other, 2
thereby producing C12 =66 comparisons. The subjects were asked to evaluate the dissimilarity of the textures for each pair. The order of the pairs was randomized. The recording method presented in [7] was used in this experiment. Every subject was given a sheet of paper with a horizontal line that contained a visual analogue scale from 0 to 10. When a pair of textures was presented to them, they perceived the textures using the method described in 3.2.2 and placed an “x” on the line according to the dissimilarity of the textures they felt. The subjects were told that they should place the “x” at “0” if the two textures were “fairly similar or identical”. If they felt that the textures were "very different" from each other, they were instructed to place the “x” at “10”. The evaluation results were normalized and represent the dissimilarity of those textures.
Step 2: To research how the combination of simulation parameters influenced the perception of the haptic texture, the subjects were asked to estimate the magnitude of the four characteristics (roughness, denseness, flatness, hardness) when each texture was presented. They drew an “x” on a piece of paper to record the sensing results by using the same method mentioned in step 1. “0” represented smooth, hard, flat and coarse, whereas “10” represented rough, soft, humpy and fine. In addition, the perceptual characteristics increased from 0 to 10. Each scale was on a separate slip of paper. The papers were passed to the subjects by the experimenter, one piece at a time. The subjects made an “x” mark on the paper representing each scale at the point at which she/he judged to be the appropriate position. All of the four scales were completed for a given texture before the experimenter proceeded to the next texture. The textures were presented in a random order. After finishing the estimates for all of the 12 textures, the subjects rested for five minutes and then repeated the experiment. During this round of experiments, the end points of the line had opposite meanings: “10” represented smooth, hard, flat and coarse and “0” represented rough, soft, humpy and fine. We work out the averages of the results of the two rounds and normalized them.
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 6
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
TABLE 2 DISSIMILARITY MATRIX ACROSS ALL 12 TEXTURES
3.2.3 Results
The responses of the participants in step 1 were scaled from 0 to 1 and arranged in a matrix, as listed in Table 2. In step 2 of the main experiment, the researchers recorded the magnitude estimates of the subjective characteristics which are presented in Table 3.
4 ANALYSIS AND DISCUSSION 4.1 Effects of Parameters on the Perceived Texture To study how the texture spatial period, texture height, stiffness coefficient and their combinations would influence the perception characteristics of the 12 textures, three-way repeated measure ANOVA was used to analyze the data collected from the 19 subjects, which are presented in Table 3. The results of the three-way repeated measure ANOVA show that there are significant effects of the spatial period, height and stiffness coefficient on the perception of roughness ([F=40.31, p
1
Experimental Study on the Perception Characteristics of Haptic Texture by Multidimensional Scaling Juan Wu, Member, IEEE, Na Li, Wei Liu, Guangming Song, Member, IEEE，and Jun Zhang，Member, IEEE Abstract—Recent works regarding real texture perception demonstrate that physical factors such as stiffness and spatial period play a fundamental role in texture perception. This research used a multidimensional scaling (MDS) analysis to further characterize and quantify the effects of the simulation parameters on haptic texture rendering and perception. In a pilot experiment, twelve haptic texture samples were generated by using a 3-degrees-of-freedom (3-DOF) force-feedback device with varying spatial period, height and stiffness coefficient parameter values. The subjects’ perceptions of the virtual textures indicate that roughness, denseness, flatness and hardness are distinguishing characteristics of texture. In the main experiment, nineteen participants rated the dissimilarities of the textures and estimated the magnitudes of their characteristics. The MDS method was used to recover the underlying perceptual space and reveal the significance of the space from the recorded data. The physical parameters and their combinations have significant effects on the perceptual characteristics. A regression model was used to quantitatively analyze the parameters and their effects on the perceptual characteristics. This paper is to illustrate that haptic texture perception based on force feedback can be modeled in two- or three-dimensional space and provide suggestions on improving perception-based haptic texture rendering. Index Terms—Virtual texture perception, multidimensional scaling, texture perceptual space
————————————————————
1 INTRODUCTION IRTUAL haptic texture rendering transmits the con-
V
tact perception of surface roughness to operators through haptic display devices. With the development of haptic texture rendering technology, virtual haptic rendering has been widely used in many areas, such as biomedical engineering [1] [2], military simulations [3] [4], art galleries [5], and teleoperation. Particularly essential applications include enhancement of the sense of reality of the blind and human-computer interaction (HCI). Prior to simulating real textures, it is essential to research the inherent mechanisms of real tactile texture perception. It is necessary to have a better understanding of the generation of texture sensation and the physical factors that influence the perception of texture. Previous research studies have contributed much to real texture perception, and its characteristics have been under research for years. Real textures can be classified into artificial textures and natural textures. The artificial textures, such as paper and silk, are created and designed by human beings while the natural textures, such as the textures of water, stone, and wood, come into being naturally. Although most all of the textures employed in ————————————————
The authors are with the Robotic Sensor and Control Laboratory, School of Instrument Science and Engineering of Southeast University, No.2, Sipailou, Nanjing, China. E-mail: 220132608@ seu.edu.cn.
xxxx-xxxx/0x/$xx.00 © 200x IEEE
previous studies are artificial textures and can be experienced by direct touching, these studies vary in terms of their mathematical approaches, methods of psychological experiment as well as the stimuli. These distinct factors resulted in different experimental findings. To explore the characteristics of real texture perception, Yoshida researched the major dimensions of touch using 25 samples with various textures [6] and discovered that the most important dimensions were heaviness, coldness, wetness, smoothness and hardness. According to Hollins et al. who used 17 tactile stimuli to research the subjective dimensionality of tactile surface texture perception [7], roughness-smoothness and hardness-softness were robust orthogonal dimensions. According to Picard et al. who researched the perceptual dimensions of everyday tactile textures [8], the perceptual dimensions were softness/harshness, thinness/thickness, relief and hardness. And according to Ballesteros et al. who analyzed the perceptual space of 20 materials, there was a two-dimensional (2D) space in which the first dimension was roughness/smoothness and the second dimension was slipperiness/stickiness and hardness/softness [9]. They also proposed a three-dimensional (3D) space in which the three dimensions were roughness/smoothness, slipperiness/stickiness, and hardness/softness [10]. Summers et al. constructed a perceptual space of 10 types of paper, and the first and second dimensions were predicted to be the roughness and stiffness, respectively [11]. Guest et al. analyzed the tactile dimensions of five types of fabrics [12] and found Published by the IEEE Computer Society
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 2
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
that the extracted dimensions included the roughness/smoothness, moisture/drought, and hardness/softness. And similar dimensions have been discovered in different body sites. The perception of real texture can also be affected by its contact mode, whether touched with a finger or with a probe. Texture perception with a probe can be influenced by roughness [13], hardness [14], and stickiness [15]. Yoshioka compared the texture perception from direct touch with that from indirect touch via a probe held in one hand [16]. He concluded that there were 3D spaces for both types of touch and identified the dimensions as roughness, hardness, and stickiness. He also discovered that the texture perceptions were similar, although not identical, in the two scanning modes. In addition, coldness/warmness cannot be conveyed by probes. Okamoto et al. reviewed the existing research regarding the tactile dimensionality of the physical properties of different materials [17]. Based on the analyzed studies, the authors concluded that the following three distinguishing characteristics composed the haptic texture perceptual space, regardless of whether the perception was obtained via a device or by direct touch: roughness/smoothness, hardness/softness, and stickiness/slipperiness. A virtual texture is rendered differently from a real one because of the performance limits of hardware and the exploration modality. Because of the point contact and kinetic characteristics of haptic textures, many researches regarding haptic textures have been performed with force feedback devices. However, much of the existing literature about virtual textures focuses on algorithms and hardware rather than perceptual characteristics or relationships between perceptual characteristics and the properties of the simulated textures. Minsky et al. [18] extended control algorithms and spring forces based on local gradients to create textured surfaces. In later research [19], they simulated haptic surface textures by using lateral forces and examined the textural perception of roughness for varied force characteristics and spatial geometries. Their results suggest that roughness is rated along a single-axis physical scale that depends on the force amplitude [19]. Ahmaniemi et al. [20] designed a haptic texture display system by using a single vibrotactile actuator and motion sensor. They concluded that the ridge length and spatial denseness influenced the perceived roughness and flatness. Unger et al. [21] performed virtual texture perception experiments with highfidelity magnetic levitation and developed a psychophysical function for roughness perception. They discovered an inverted “U”-shaped function between the roughness and the inter-element spacing that was nearly identical to that of real surfaces. Colwell et al. [22] used a magnitude estimation method to research the roughness of ten simulated textures, and they found the crucial relationship between the roughness perception and the simulated physical parameters. These previous studies of virtual texture forces and their perception are fruitful. However, the results of these studies are not comparable because of their diversity in
terms of facilities, modeling approaches and data analysis methods. Most of the studies concentrate on one specific perceptual characteristic of virtual texture, especially the roughness. They revealed that the perception of roughness was affected by the physical parameters. For example, Ahmaniemi researched the effects of envelope ridge lengths, spatial densities, and regularity on the perceptual characteristics of a virtual texture. The haptic texture was generated by a single vibrotactile actuator and motion sensors instead of force feedback devices. However, the relationships between the actual perceptual characteristics other than roughness and the simulated parameters of force-based virtual textures have not aroused the researchers’ interest yet. This paper intends to analyze the relationship between the perceptual characteristics and the simulated parameters based on general force feedback devices, which is necessary to uncover the inherent connection between the input and output of haptic texture rendering systems and to make haptic rendering systems more realistic. To clarify the perceptual characteristics of virtual textures generated by force feedback, the research team applied a general 3-degrees-of-freedom (3-DOF) force feedback device to create tactile force stimuli and try to reveal the virtual haptic texture perceptual space with a basic and simple haptic texture model. A standardized periodic texture signal was used to simulate the tactile sense, similar to regular textures. The haptic texture was simulated when a user manipulated the force feedback device so as to help it move over the surface of the virtual texture. The multidimensional scaling (MDS) method was used to analyze the experiments. From the perspective of psychophysics, the researchers further explored how objective parameters affected human texture perception and how people distinguished different haptic textures. The relationship between the perceptual characteristics of the virtual texture and the objective parameters has also been studied. As a result, some valuable suggestions for further improving the haptic texture rendering based on force feedback will be provided.
2 SYSTEM AND METHODS 2.1 MDS Analysis MDS is a method to measure the similarity (or dissimilarity) among pairs of objects as distances between points in a low-dimension multidimensional space [23]. The output of the MDS method is a configuration of objects embedded in a multidimensional space. The MDS technique has been widely applied in tactile and haptic perception studies [24] [25] [26] [27]. The data analysts can observe the data and display according to the graphical display of the correlations provided by MDS [28]. MDS analysis can provide information concerning the psychological representation of objects, including the dimensionality of their perceptual space, interpretation of their dimensions, and configuration of the objects in their perceptual space. The K-stress and RSQ are applied to determine the fitness between the dimensionality and the observed data. The K-stress is a normalized difference between the fitted dis-
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics AUTHOR ET AL.: TITLE
3
Fig. 2. Schematic of the haptic interaction algorithm.
Fig. 1. (a) A real texture perception system based on point contact.
Fig. 3. Schematic of the texture height used in the experiments.
to control the virtual ball at a specified velocity. The speed was kept constant so that the results could exclude the effects of varying speed [30] [31].
2.3 Texture Force Rendering Algorithm
Fig. 1. (b) The virtual texture perception system used in our experiments, which simulates a point contact with a real texture surface.
tances and the observed proximities. Stress values of less than 0.2 and a sudden decrease in the stress values on the plot indicate that the output fits the similarity data well [29]. In this study, human perceptual dissimilarity ratings of a set of haptic textures were used as the input data. Stress values were calculated for one- through- fivedimensional solutions. According to the stress values and the trend of the plot, output configurations were obtained; these configurations could be interpreted as maps of the haptic textures in a psychological space.
2.2 Experimental Design The system applied in our experiments is as shown in Fig.1, of which, Fig.1. (a) is a real texture perception system used in [16], while Fig.1. (b) is our virtual texture perception system. In contrast with the real texture perception system, the virtual system uses the relatively inexpensive, portable, safe, and reliable 3-DOF Omni force feedback device manufactured by SensAble Corp. (located in the city of Wilmington, Massachusetts) for texture display. The force feedback device has a position resolution of 0.055 mm and can provide a maximum force of 3.3 N. In the real texture perception system, an operator placed a probe on a real texture. In the experiment, a user held the Omni probe to make the haptic interface point (HIP) move back and forth on the textured surface. The appropriate force was generated and pushed back to the user to simulate a collision through the probe. In the experiments, subjects followed the target ball on the screen
A texture force can be modeled based on physical measurement, geometrical constraints and a physical model or image processing [32][33][34]. Previous experimental studies suggest that the main characteristics of the real textures are roughness/smoothness, hardness/softness, stickiness/slipperiness, and flatness/bumpiness. The main factors that influence those characteristics include the spatial period, height and stiffness [35][21][36][37]. Therefore, to synthesize the haptic texture, the researchers chose the waveform, spatial period, height and stiffness coefficient as the parameters. As the focus of this paper is the physical factors of texture surface, the texture force model is simplified and only related to the physical properties. The virtual surface is idealized as a regular square wave. The HIP is inside the virtual object when the device comes in contact with the object, as shown in Fig. 2. The center of the circle represents the HIP. The force is perpendicular to the associated surface, and the magnitude is a function of the vertical distance from the surface to the HIP. The force is calculated and actuated according to Hooke’s law, a simple linear force model: F=ky (1) where k is the stiffness coefficient and y is the vertical distance. The texture is created by the 3-DOF Omni force feedback device. To simplify the analysis and eliminate the other factors which would influence the experimental results, the research used an ideally textured surface that was similar to that used in [25]. The texture force model was an idealized square wave signal that was perpendicular to the virtual texture surface. The researchers established a 3D space of the virtual texture with the Cartesian space, which was defined by the Oxyz coordinates. The texture height hZ was along the z axis. It was perpendicular to the
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 4
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
texture surface and changed periodically with changes in the x-axis coordinate, as shown in Fig. 3. The y-axis is not shown in the figure because the height do not change in the y direction. The height of the texture can be expressed as
ℎ =
ℎ, sin(2 0, sin(2
/ /
) > 0 )≤0
(2)
where is the horizontal axis (x-axis) coordinate of the texture surface and ℎ is the height of the square wave, denotes the spatial period. The which is constant, and duty cycle ratio is 1:1. Except the tangential friction, the feedback function of the texture force can be expressed as
=
(ℎ − ), ≤ℎ 0, others
(3)
is the amplitude of the texture force, is the Here, position of the HIP on the z-axis, and k is the stiffness coefficient. When the handle moves into the texture plane, ≤ ℎ , there is an output of force feedback. where The texture parameters of the texture force model in our research were set as follows. According to our previous study, spatial periods in the range of 0.5-15 mm have a significant effect on the perception of the texture force [38][39]; thus, the three spatial periods d0 used in the experiments were within this range: 0.5 mm, 6.9 mm and 15 mm. Then, according to the work of Lawrence et al. and Liu [37] [40], along with consideration of the range of stiffness of the device, the height of the haptic texture h was set to 5 and 25 mm, and the stiffness coefficient was set to 0.1 and 1.0 N/mm. Using all possible combinations of these parameters yields 12 different haptic textures, as listed in Table 1.
3 TEXTURE PERCEPTION EXPERIMENTS 3.1 Pilot Experiment Based on previous studies, the texture spatial period, texture height and stiffness coefficient were chosen as the objective stimulus parameters, and 12 virtual textures were synthesized. The purpose of the pilot experiment was to clarify how subjects would describe the haptic TABLE 1 12 VIRTUAL TEXTURES USED IN THE EXPERIMENT
texture when no guidelines or constraints were provided by the experimenters, i.e., the researchers expected to confirm whether the haptic texture reflected the characteristics of the real texture. The goal of this experiment was to provide some insights into the identity of different surface textures and to help discover the differences between the perception of real and virtual textures when the same objective parameters were applied. The experiment would also yield a list of characteristics to describe the feeling of the textured surface.
3.1.1 Participants Eight subjects participated in this experiment. Their ages ranged from 20 to 30, all of whom were right-handed and reported to have no known cutaneous or kinesthetic problems.
3.1.2 Procedures
The subjects sat at the experiment table with a computer screen and the Omni force feedback device placed on it (see Fig. 1. (b)). In each trial, the subject moved the probe of the Omni across a textured surface and felt the virtual texture. Then, he or she described what he or she felt about those textures after or during the experiment. Twelve virtual textures were presented in a random order.
3.1.3 Results
The subjects provided the following descriptions. Participant 1: smooth, concave-convex, rough, soft, hard. Participant 2: smooth, wide. Participant 3: frequency-dependent, magnitude of the force, rough, smooth, height. Participant 4: amplitude, frequency, duty cycle. Participant 5: magnitude of the force, rough, concaveconvex. Participant 6: rough, smooth, level of fluctuation. Participant 7: periodicity, concave-convex. Participant 8: soft, hard. The researchers classified the words provided by the subjects into roughness (smooth and rough), denseness (wide, frequency-dependent, frequency, duty cycle, and periodicity), flatness (concave-convex, height, amplitude, and level of fluctuation) and hardness (soft, hard, and magnitude of the force). Denseness is a characteristic that is related to the spatial period and can be expressed as coarse or fine. Roughness (smooth/rough), hardness (hard/soft) and flatness (flat/bumpy) are characteristics that most people are familiar with. Comparing those characteristics with real texture perception, we observed that the haptic textures created by combinations of those objective parameters had almost the same characteristics as real textures. Then, in the main experiment, the subjects were asked to evaluate those characteristics for all 12 textures. Through the MDS method, the characteristics used by people to distinguish different virtual textures in different perceptual spaces were studied.
3.2 Formal Perception Experiment The formal perception experiment was conducted using
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics AUTHOR ET AL.: TITLE
the 12 virtual textures. The purpose of the experiment was to study the correlation between the input and output of the system and to determine whether these parameters and perceptual characteristics provide a useful method for interpreting the model of the haptic texture perceptual space. Adjustable simulation parameters, including the spatial period, height and stiffness coefficient, were recorded in the system. The perceptual characteristics including roughness, denseness, flatness and hardness were observed. In the experiment, textures with different combinations of the adjustable simulation parameters were presented to the subjects. A scale from 0 to 10 was adopted to assess the perceptual differences of those textures and the magnitude of each characteristic. The experiment was conducted in two sessions. In the first step, a dissimilarity matrix that distinguished between the perceived dissimilarities of the two stimulus surfaces was observed, where the MDS method was used to establish a satisfactory perceptual space on the basis of this dissimilarity matrix. And in the second step, subjects were asked to provide a quantitative description for each characteristic on a scale from 0 to 10, where three-way repeated measure ANOVA was used to analyze the correlation between the stimulation parameters and perceptual characteristics. Because the objective parameters and perceptual characteristics were the inputs and outputs of the system respectively, the researchers studied the composition of the dimensions from those two aspects. Then, the quantitative relationship between the input and output and the orthogonality of the subjective perceptual characteristics were analyzed. These two steps enabled the researchers to identify the basis by which people distinguish between different virtual textures.
3.2.1 Participants Nineteen subjects who ranged from 20 to 30 years old participated in this experiment. None of them had participated in the pilot experiment. And all were right-handed and reported to have no known cutaneous or kinesthetic problems.
3.2.2 Procedures
The purpose of the main experiment was to obtain data that indicated the perceived dissimilarity of the textures and evaluate the characteristics of each texture. To exclude the influences of the exploratory speed, a constant speed was applied for the perception of different subjects and different textures. According to the previous studies and the habitual exploratory speed of different subjects, 50 mm/s was selected for the experiment. All of the subjects needed some practice to maintain the constant speed by following the target ball on the screen. In the experi-
5
ment process, the subjects were allowed to explore the stimulus in the horizontal direction by scanning back and forth until they were satisfied. Subjects were instructed to report their initial impression of the virtual textures rather than trying to determine the “correct” answer.
Step 1: The 12 textures were compared with each other, 2
thereby producing C12 =66 comparisons. The subjects were asked to evaluate the dissimilarity of the textures for each pair. The order of the pairs was randomized. The recording method presented in [7] was used in this experiment. Every subject was given a sheet of paper with a horizontal line that contained a visual analogue scale from 0 to 10. When a pair of textures was presented to them, they perceived the textures using the method described in 3.2.2 and placed an “x” on the line according to the dissimilarity of the textures they felt. The subjects were told that they should place the “x” at “0” if the two textures were “fairly similar or identical”. If they felt that the textures were "very different" from each other, they were instructed to place the “x” at “10”. The evaluation results were normalized and represent the dissimilarity of those textures.
Step 2: To research how the combination of simulation parameters influenced the perception of the haptic texture, the subjects were asked to estimate the magnitude of the four characteristics (roughness, denseness, flatness, hardness) when each texture was presented. They drew an “x” on a piece of paper to record the sensing results by using the same method mentioned in step 1. “0” represented smooth, hard, flat and coarse, whereas “10” represented rough, soft, humpy and fine. In addition, the perceptual characteristics increased from 0 to 10. Each scale was on a separate slip of paper. The papers were passed to the subjects by the experimenter, one piece at a time. The subjects made an “x” mark on the paper representing each scale at the point at which she/he judged to be the appropriate position. All of the four scales were completed for a given texture before the experimenter proceeded to the next texture. The textures were presented in a random order. After finishing the estimates for all of the 12 textures, the subjects rested for five minutes and then repeated the experiment. During this round of experiments, the end points of the line had opposite meanings: “10” represented smooth, hard, flat and coarse and “0” represented rough, soft, humpy and fine. We work out the averages of the results of the two rounds and normalized them.
1939-1412 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TOH.2015.2438866, IEEE Transactions on Haptics 6
IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID
TABLE 2 DISSIMILARITY MATRIX ACROSS ALL 12 TEXTURES
3.2.3 Results
The responses of the participants in step 1 were scaled from 0 to 1 and arranged in a matrix, as listed in Table 2. In step 2 of the main experiment, the researchers recorded the magnitude estimates of the subjective characteristics which are presented in Table 3.
4 ANALYSIS AND DISCUSSION 4.1 Effects of Parameters on the Perceived Texture To study how the texture spatial period, texture height, stiffness coefficient and their combinations would influence the perception characteristics of the 12 textures, three-way repeated measure ANOVA was used to analyze the data collected from the 19 subjects, which are presented in Table 3. The results of the three-way repeated measure ANOVA show that there are significant effects of the spatial period, height and stiffness coefficient on the perception of roughness ([F=40.31, p
