I. Introduction
Recently, fashion consumers have increased opportunities to access information about a product on the screens of their smart phones or tablet PCs as opposed to directly touching or seeing the product. Moreover, in the fashion industry along with the development of ICT, it is becoming more common to have experience with indirect fashion images, where one acquires information on a screen rather than through direct product experience. This is done through fashion films, 3D virtual garment simulations, and virtual shopping and by gaining fashion image experience using SNS. Such a trend is being increasingly activated and accelerated with the advent of various shopping channels using online and AR/VR technologies, and it has become more important to judge and evaluate a haptic sense of fashion materials; that is, the most important attributes of fashion products depend on memories or visual experiences.
In particular, because consumers think highly of wearing a fashion product directly and touching or feeling the material, haptic feedback can have a core effect on product evaluations and purchase decisions (Jeong, Jang, Chae, Cho, & Salvendy, 2008;Peck & Childers, 2003). In a situation in which shopping for fashion products is done using digital media, consumers perceive uncertainty and risk intensively, and the absence of material information or a difference from actual material information will have a negative effect on their intentions to purchase, increasing the rate of returned goods (Estlick & Feinberg, 1999;Kim & Lennon, 2000;Na, Lee, & Shin, 2011). Particularly, Hammond and Kohler (2002) found that information about color, a sense of touch, and size as factors causing consumers to perceive difficulties and held that the most important issue is to express haptic information effectively and accurately. When one experiences a fashion product through images, haptic sense plays a very important role, and haptic experience is most of all a very important element in experiencing the physical attributes of fashion products in a virtual environment.
Online shopping is now common, with many consumers experiencing fashion product images in a virtual environment using VR and AR. Hence, the development and application of haptic technology is now required for accurate communication of the touch sense of fashion products to users. In other words, it is necessary to develop a haptic communication system for fashion images capable of conveying the material quality of a fashion product in a realistic and authentic manner. However, precedent studies on haptic technology focused on the development of systems that provide touch sense in engineering fields, with the majority related to technological development to provide a sense of friction when one stretches their fingers. The number of applied fields is also limited (Khasnobish, Pal, Tibarewala, Konar, & Pal, 2016;Tanaka, Goto, & Sano, 2017;Wu & Smith, 2015), and although there have been studies of fashion shopping and fashion product applications in VR and AR (Hur, Jang, & Choo, 2019;Kim, Park, Kim, & Choi, 2020), earlier work on the delivery of touch feedback information of a fashion product using haptic technology is very rare.
The goal of this study is to develop a haptic communication system that can convey tactile sensations of fashion materials in a virtual environment and to verify how realistically and effectively the virtual fashion fabric image of this system delivers the tactile sensation of actual fashion fabric. To do this, the following research questions were established. First, a questionnaire for evaluating the tactile attributes of fashion materials was developed. Second, a haptic communication system for fashion image experience in a virtual environment was designed and a haptic rendering model was suggested. Third, the effectiveness of the developed haptic communication system was verified through a user evaluation. Through a literature review, the tactile attributes of fashion materials were defined for implementation through the haptic communication system, and questions were developed to evaluate them. Based on this, the haptic communication system was designed. Through the user evaluation, the validity of the tactile attribute evaluation questions of the fashion materials and the effects of the haptic communication system were verified.
The results are expected to provide suggestions for researcher to develop interactive content using the haptic technology, which can increase immersion in various fields, including medicine, education, entertainment and the arts, as well as fashion goods. The results here are also expected to provide ideas about how multisensory information can be communicated to users through fashion images.
Ⅱ. Background
1. Attributes of fashion materials for haptic sensing
Haptic sense is a feeling related to kinesthetic and tactile sensations. Kinesthetic feeling refers to sensory data, which is obtained through receptors of the joints, muscles, and ligaments. These feelings include softness, tenderness, hardness and stiffness. Tactile feeling is sensory data obtained through receptors of the skin in the form of distributed pressure, vibration, temperature and pain (Davis, 1987). In other words, tactile information refers to the primary sensations of an object surface as felt on one’s skin, and tactile sense pertains to comprehensive sensitivity consisting of the material attributes, texture, construction and of prior experience or associations with similar objects (Lee & Kim, 2016). Such haptic experience, i.e., the sense of touch, is not realized only by the action of one’s skin touching the surface of an object but also comes from softness, tenderness, hardness, volume, stiffness, elasticity and a sense of temperature via the actions of pressing, grabbing, rubbing and twisting with one’s fingers (Lee & Kim, 2016).
To define the attributes of fashion materials for haptic sensing as delivered by haptic communication, a review of the literature shows that the ASTM (American Society for Testing and Materials) classifies fashion material’s physical attributes into skin unevenness, skin friction, density, hot and cold, extensibility, compressibility, resilience and flexibility. In this respect, Davis (1987) divided the attributes of surface materials considering the skin type, skin friction, hot and cold, and haptic attributes in the event of touching with the hand into flexibility, compressibility, elasticity, resilience and density. Choo and Kim (2002) defined gloss, unevenness, thickness, and density as surface attributes of fashion materials. Regarding the influence of the sense of sight and touch and the individual senses of sight and touch on subjective evaluations of fabric by hand, Lee and Kim (2004) mentioned the five factors of the feelings of unevenness, density, flexibility, moisture and elasticity as haptic senses felt when touching a fashion material. Also, in a study of the sensing of visual tactility of modern fashion materials, Lee and Kim (2016) suggested feelings of unevenness, density, flexibility, gloss and transparency. Among the aforementioned seven factors (unevenness, density, thickness, hot and cold, flexibility, elasticity, and resilience) of fashion materials, four, specifically density, thickness, flexibility and elasticity, can be detected using kinesthetic sensing.
These four factors of the kinesthetic sensing of fashion materials are implemented into a haptic communication system. Density refers to a value obtained by dividing the mass of a substance by the volume, and for fabric it refers to the number of warps and the weft per unit length. Materials of high density where the strands per unit length are dense include heavy materials when compared at a certain volume. Thickness refers to the degree of a material’s thickness and influences the strength and air permeability, and elasticity is related to the capability of a substance to stretch and shrink, also called resilience. A material that stretches and changes easily even with a slight amount of external force has good elasticity. Flexibility refers to the degree of softness, as in highly flexible materials that are easy to bend. An excessively flexible material loses its strength and does not easily maintain the form of clothes because it can be dropped (Kim, 2009).
In the existing textile material research field, many studies have sought to determine how physically to measure the tactile elements of fashion materials through experiments, whereas research on the scale and question items that can be used to evaluate tactile sensations delivered to actual users has yet to be conducted. Therefore, in order to verify the effectiveness of the haptic communication system in this study, it is necessary to conduct a study on tactile evaluation items. To this end, based on existing definitions of the physical properties and tactile elements of fashion materials (Choo & Kim, 2002;Davis, 1987;Kim, 2009;Lee & Kim, 2004;Lee & Kim, 2016), questions for evaluating the tactile properties of fashion materials are presented in <Table 1>.
2. Haptic communication system and rendering for fashion images
This research has three steps. First, a means by which to evaluate the tactile properties of fashion materials was developed. Second, the haptic communication system was designed, and third, the haptic communication system was verified. The three research questions in this study were verified by evaluating the differences between fashion images experienced in a virtual environment and those of actual fashion materials as experienced through a haptic communication system for fashion images. The haptic communication system for the fashion image experience refers to the configuration with which to interact with visual and haptic components through fashion images. Visual and haptic interaction with fashion images was achieved by constructing a haptic communication system. A haptic device interface exists to convert a visually implemented fashion image to a haptic experience, and a graphic rendering system and haptic rendering system are designed to interact through this haptic device. To realize this, the haptic communication system was designed and a haptic rendering model for fashion images was proposed. These systems are described below.
1) Haptic communication system for fashion images
Visual and haptic interaction with fashion images was achieved by constructing a haptic communication system. The developed haptic communication system is composed of a visual interface device (a monitor), a haptic interface device, and a virtual environment, as shown in <Fig. 1>. The fashion images are displayed in a virtual environment. When a user grasps a haptic interface device and moves it, the position of the haptic interface point (HIP) is conveyed to the virtual environment. As the position of the HIP collides with the fashion image in the virtual environment, the haptic rendering method computes the deformation of the image and the feedback force according to the amount of user interaction with the fashion image and the corresponding mechanical properties. The computed feedback force is conveyed to the user via the haptic device and the computed deformation is realized on the visual interface device through graphic rendering. To visualize and hapticize the fashion image in rendering modules, the Open GL library (3D graphics library) and the Chai3D library (haptic library) were used. The experiment is conducted with a Geomagic Touch device, which provides three DOFs of force feedback and precisely senses six DOFs of positions (with a resolution of .055 mm). We simulated the virtual fashion images at a haptic update rate of 1000Hz and at a graphic update rate of 60Hz in order to obtain robust and continuous graphic and haptic information. During the haptic rendering process, the haptic behavior of a fashion image and its deformed shape are computed using a mass spring model.
2) Haptic rendering for fashion images
As noted above, haptic rendering for fashion images was done to convert visually implemented fashion images into haptic experiences, and the fashion images were represented by a mass-spring-damper model (as a haptic system) consisting of point masses (nodes), dampers, and springs, as shown in <Fig. 2>. In the figure, the small spheres (colored in blue) are the point masses, and the connection between a point mass and its neighbor is realized using a spring and a damper. The springs exert force on the nodes when the nodes are displaced from their resting positions. In this work, four major material properties (density, thickness, flexibility and elasticity) of a target fabric were haptically simulated by the mass-spring-damper model, whose governing matrix equation is expressed as follows:
In (1), M is the mass matrix, C is the damping matrix, K is the stiffness matrix, x is the nodal displacement vector, and f is the external force vector. By computing (1), we can easily calculate the feedback force when the HIP collides with the virtual fashion fabric image. The computed force vector is conveyed to the haptic device. Therefore, a user can haptically sense the virtual fashion fabric image as the user can feel an actual fabric.
As noted above, an update rate of 1 kHz is required to ensure stable haptic rendering. Furthermore, we attempted to increase the update frequency of the graphics rendering to 60 Hz because a user feels the material as much smoother than at 30 Hz. A multithread technique was adapted to achieve a high update frequency for both haptic rendering and graphics rendering, as shown in <Fig. 3>. In the haptic thread, the position of the HIP is determined in order to compute the collision detection between the HIP and the other virtual objects (the fashion image or a wall). Moreover, the position of the HIP is conveyed to the graphics thread to update the position of the interaction tool visually (e.g., a fingertip or an awl, among other items). If a collision between the HIP and the virtual fashion image occurs, the haptic system based on the mass-spring-damper model calculates the feedback force and positions of each node, after which those results are updated. The calculated feedback force is transmitted to the haptic device and the computed position of each node is conveyed to the visual interface device. Thus, a user not only sees the deformed shape of a fashion image but also haptically feels a variety of mechanical properties of the image.
Ⅲ. Methods
A user study was conducted to evaluate the ability of the studied haptic effect to simulate fabrics, included user evaluation of the proposed haptic communication system. The experiment aimed to compare differences during the haptic sensing of fashion materials using the haptic communication system.
1. Participants
This experiment involved adult male and female graduate students majoring in fashion in their 20s-30s who are living in Korea. To verify the differences between tactile sensing by hand and by haptics, graduate students majoring in fashion with a high understanding of fashion materials and images were assumed to be adequate, and 5 males and 15 females were selected. Before participating in the experiment, they had neither information about the experiment nor experience with haptic devices.
2. Experiment environment
The proposed haptic communication system was evaluated through four virtual applications as shown in <Fig. 4>, for comparing the differences between the transmitted feeling of the virtual fashion material through the haptic communication system and touching the actual fashion materials by hand. We constructed three virtual environments in which a user can haptically sense the density (Fig. 4(a)), thickness (Fig. 4(b)), elasticity and flexibility (Fig. 4(c)) of a fashion material. <Fig. 4(a)> shows an application during which a user haptically feels the density of a fashion image by punching it using an awl. A fashion image with higher density is more difficult to pierce. We implemented another virtual environment in which a user can haptically sense the thickness of a fashion image in view of the fact that a human haptically identifies the thickness of a fabric by pressing it (Fig. 4(b)). Haptic feedback in this case depends on the mechanical properties of the fashion image and the corresponding pushing depth. In <Fig. 4(a)>, we fixed all four corners of the fashion image, and we fixed the lower surface of the image in <Fig. 4(b)>. In the real world, the elasticity of an object can be understood by pulling on a fabric. In order to simulate a fashion image in a virtual environment haptically, the two upper corners were fixed. During this experiment, a user grasped a portion and pulled the fashion image to understand the elasticity of the fashion image, as shown in <Fig. 4(c)>.
3. Materials
Materials samples selection for evaluation was carried out as follows. First, 26 types of fabrics generally used as fashion products were collected through reviews by three fashion professionals. The collected fabrics were assumed to render one experience well, i.e., the density, thickness, elasticity, or flexibility. These pertain to factors of the tactile sense of the fashion material to implement in the haptic communication system, and the 15 participants were requested to evaluate each fashion material’s tactile sense of touch. Four factors of tactile sense of fashion materials were evaluated through grasping, rubbing, or touching a fashion material on their skin, with responses recorded on a Likert scale. Participants had to enter a score of one for the thinnest material and seven for the thickest material. For each factor of the tactile sense, materials with the highest scores were chosen, and pairs of fabrics with the highest and the lowest density, thickness, elasticity and flexibility scores were selected. Finally, six types of samples include those overlapping for each element were determined as well designed for the participants to experience the tactile elements of fashion materials through a haptic communication system.
The Korea Apparel Testing and Research Institute and the Seoul National University Research Institute of Advanced Materials were asked to measure the fiber and structure blending percentages, the density, the thickness, the extension recovery rate, the tensile strength, and the flexural strength of each material sample used in the evaluation, as indicated in <Table 2>. The study aimed to obtain objective physical data pertaining to general textile properties, including the fiber and structure blending percentages and the density and the thickness of the material samples for evaluations by KATRI. At SNURIAM, measurements were taken of the tensile strength and flexural strength, which will allow comparisons of the mechanical characteristics of the flexibility and elasticity by the Kawabata system. LT, pertaining to the tensile strength, means linear tensile and is the tensile degree of a material. The closer a value is to 1, the harder a fabric is, with poorer elasticity. WT refers to the tensile energy per unit area, and a larger value in this case indicates an easier fabric to pull, with better elasticity. B is the value of the bending rigidity used for measuring the flexural strength. The smaller this value is, the softer and more flexible the fabric is.
4. Experiment design
Data for the tactile sensing of the fashion materials designed for the experiment were inputted with ad-justments of the parameter values of the damping, stiffness and mass. There were no references for the parameter values of the damping, stiffness and mass to express the fashion materials. Accordingly, default values were set based on the experience of the researcher and the final data were set as follows via reviews by three experts, as indicated in <Table 3>. Damping is a parameter that adjusts the degree to move flexibly, and it shows movement closer to water as the value becomes lower and closer to melted cheese as the value becomes higher, showing the need for a proper adjustment in the elasticity and flexibility modes. Stiffness refers to the ability to stretch and shrink well, and an adjustment was mainly made in a thickness mode, as the material is harder when the value is higher. Regarding the mass, the higher the value is, the heavier the cloth on the screen becomes. Hence, the design included an adjustment of a function to fix a gimlet in a mode for measuring the density.
5. Experiment procedure
This research involved human subjects and interactions, including communication and interpersonal contact. It was reviewed by the authorized institutional review board (IRB No. 1810/003-019) and approved in terms of its ethical and scientific validity and on the measures of the privacy of the participants. The research participants were also given a manual, and agreement was obtained from them before the experiment, and the research aims and procedures were explained.
Before the participants began the experiments, they were instructed regarding the purpose, procedure, and order of the tasks of the study. Participants were provided with the opportunity to learn and practice as much as they wanted before the experiment. The experiment was conducted on a one-on-one basis for about one hour over five days from the 22nd to the 26th of October of 2018 at a soundproof and restricted laboratory.
An experiment was conducted to compare tactile sense experiences with an actual fashion material and that from a haptic device. The experiment was composed of two steps. First, the participants were asked to experience an actual fashion material by hand motions, including grasping, pulling or stretching, and to evaluate the degree of the density, thickness, elasticity and flexibility they felt on a seven-point Likert scale. Participants sat on a chair in front of a monitor and operated Phantom Omni, a haptic delivery device while staring at the monitor. When the researcher changed the screen in the order of the experiment, the participants experienced fashion materials on the monitor using the haptic device and evaluated the degree of the density, thickness, elasticity and flexibility they felt on a seven-point Likert scale. <Fig. 5> shows the setting of the experimental environment, and a factor analysis and paired t-test were conducted on the data collected using SPSS 23.0.
Ⅳ. Results and Discussion
1. Verification of items to evaluate the tactile properties of fashion materials
Varimax rotation of the factor analysis was conducted for the items used to evaluate the tactile sense attributes of the fashion materials. Varimax rotation is useful to understand hypothetical constructs by extracting relationships among numerous variables with correlations into a few independent factors (Rhee & Chung, 2010). The results of the principal component factor analysis are as indicated in <Table 4>.
Regarding the items for evaluating the tactile sense attributes of the fashion materials, the three factors of the density, thickness, and elasticity were extracted. Eigenvalues of three factors were 1.806 for the density, 2.654 for the thickness and 3.294 for the elasticity, and only factors with an Eigenvalue of more than one were extracted, explaining 77.573% of the entire evaluation. Item 3, Density, and Item 2, Flexibility, were eliminated because they were not grouped into a proper factor after the factor analysis. In particular, the flexibility factor was not distinguished clearly from the elasticity factor during the tactile sensing evaluation and was therefore grouped into a single factor which was termed elasticity. When the three factors of the density, thickness and elasticity were set, item content by factor, the number of items by factor, and the factor loading showed the greatest stability. Accordingly, as a result of the final factor analysis after omitting the flexibility factor, the factor loading was in the range of .696 to .940 and the Cronbach α value exceeded .837 in every case, ensuring internal consistency. The factor analysis to verify the evaluation of the tactile sense attributes of the fashion materials clarified the density, thickness and elasticity.
2. Haptic communication system efficacy
A paired t-test was conducted to verify the homogeneity between the tactile senses of the fashion materials when experienced by hand and by haptic communication. Null hypothesis H0, “Tactile sense by hand and by haptic communication is identical,” was established and the results of the paired t-test are as follows at a significance level of 5%.
Results of paired t-test as indicated in <Table 5>: Regarding the thickness and density, the p-value exceeded .05 and the mean difference is not significant; therefore, the null hypothesis is accepted. Under the rule stating that ‘if p-value ≤α, H0 shall be rejected’, it is possible to determine whether the null hypothesis is rejected or not. In the case of elasticity, the mean difference is approximately 8.85 and it appears to have increased during the haptic experiment with a standard deviation of 6.53601. Moreover, the t-value of the test statistics was 6.055, and the p-value of .000 shows that the difference between the actual tactile sense information and the haptic information is statistically significant. This indicates that for the density and thickness, the actual tactile sense information and haptic information correspond to each other, and it can be regarded that the experience of tactile sensing by the haptic device led to greater perceived reality. For the elasticity, it was found that the tactile sense experience of a fashion material by haptic communication and the actual tactile sense experience were not identical.
A user evaluation was conducted to verify the homogeneity of the tactile sense experience of the fashion materials by hand and by haptic communication. A factor analysis to verify the evaluation of the tactile sense attributes of the fashion materials indicated the factors of the density, thickness and elasticity. As a result of comparing the tactile sense by the haptic device and by hand, it was found that for the density and thickness tactile sense experiences by the haptic communication system led to greater perceived reality, though for elasticity, it was not identical to the actual experience.
The early stage of the research was designed to compare the physical and haptic data of a fashion material sample. In the user evaluation, when the partic-ipants experienced fashion materials by haptic communication, the degree of the density, thickness and elasticity of each sample they felt was evaluated on a Likert scale ranging from 1 to 7. By comparing the data acquired above with the data from the actual physical properties of the fashion material samples, this study attempted to find a value corresponding to haptic data. For this reason, repeated measures of the regression analysis had to be conducted. However, because tactile sensing shows greater individual differences, it is necessary to set an individual reference point, calculate the variation of the tactile sense according to the sample, and to identify the level of such changes. This required a scale for the tactile sense changes delivered by the haptic communication system. However, there is no such scale to deliver the tactile senses of fashion materials in earlier works. Therefore, the experimental design was revised to evaluate the homogeneity of tactile sensing by the haptic communication and by hand. It was found that for the density and thickness, the tactile sense of an actual fashion material is delivered by haptic communication similarly to touch by hand. Such a result may have originated from an error which occurred during the process of converting the data of the tactile sense factors of the fashion materials to haptic data. There was no reference to a model for haptic data conversion which could be applied to actual fashion materials; thus, data randomly set by the researcher were used, which may have decreased the accuracy. Despite reviews by three experts, it cannot be said that a systematic model for converting haptic data was developed here. Therefore, follow-up studies should develop a conversion system for haptic data applicable to fashion material images.
Ⅴ. Conclusion
With regard to indirect fashion image experience, acquiring information on a screen has become common instead of direct product experience. Therefore, this study aimed to suggest haptic experience as a means of increasing the perceived reality of haptic sense experience through visual means. In particular, for fashion products, evaluating the haptic sense of fashion materials has an important effect on product evaluations and purchases. Hence, it is regarded that haptic experience with a fashion product in a purchasing situation online in a virtual environment may provide significant clues with regard to haptic sensing. Hence, the study implemented a haptic communication system for fashion images and compared it with haptic sense experience by hand. Through this process, an attempt was made to verify the ability of the haptic communication system to deliver a sense of reality.
The study set haptic sense factors of fashion materials which can be experienced through haptic communication, in this case the density, thickness, elasticity and flexibility, and selected six types of actual fashion materials on which to conduct research. It realized the haptic communication system, which allows one to experience the density, thickness, elasticity and flexibility, using a Phantom Omni, which delivered force feedback and converted the tactile sense data of the six materials to haptic data by adjusting the parameter values of the damping, stiffness and mass.
A user evaluation was conducted to verify the homogeneity of the tactile sense experience of the fashion material by hand and by haptic communication. The factor analysis to verify the evaluation of the tactile sense attributes of the fashion materials found the factors of the density, thickness and elasticity. As a result of comparing the tactile sense by the haptic device and by hand, it was found that for the density and thickness, the tactile sense experience given by the haptic communication system led to greater perceived reality, though for elasticity, it did not match the actual experience.
This study has significance as leading research in this field given the design and implementation of a haptic communication system for fashion images, and it is meaningful because the factor of controlling haptic rendering, which can transform the tactile elements of fashion materials, was confirmed. However, a specific calculation formula and scale should be developed through subsequent research. In addition, it is necessary to modify the haptic communication system for experiences with elasticity and flexibility. In particular, elasticity and flexibility exist on a different scale conceptually in fashion material studies, but they can be considered as concepts undivided clearly in correspondence to the material’s physical factors and the development of evaluative indices. Elasticity refers to how an object stretches and contracts; it can be experienced by assessing how well the material stretches or recovers after it is pulled. Flexibility refers to the softness of the material, which means that if the fabric is easy to bend, it has high flexibility. The characteristics of stretching or bending are all related to the movement of the fabric, and the haptic communication system is designed to provide experience with the material while pulling and shaking it when it is visible on the screen. Moreover, as shown in <Fig. 3>, these two characteristics are set to be evaluated in one mode in the haptic communication system. Therefore, it is believed that elasticity and flexibility, which can be experienced through the movement of the fabric, could not be clearly distinguished. Furthermore, because the participants in the experiment were experts not in materials research but in fashion, it is regarded they had difficulty distinguishing these subtle differences via the haptic experience. In addition, a difference between the theoretical understanding of the concept of the characteristics of fashion materials and the sensational perception through touching was expected. Therefore, further in-depth research in the field of basic haptics focusing on perceptual haptics and tactile haptics is required. Accordingly, follow-up studies are needed to examine the physical factors of materials corresponding to elasticity and flexibility and to apply them to the haptic communication system. Further research can also improve how the model of haptic communication delivers elasticity and flexibility by designing an asymmetric mass-spring-damper model capable of representing the weaving. In addition, this study is a preliminary study examining the possibility of developing a haptic communication system for fashion image experience involving the design and evaluation of a haptic communication system. Therefore, because the data obtained from 20 participants here may be limited given the lack of deep research, it should be supplemented through further studies.
This study has significance in that it is the first to design and develop a haptic communication system to provide users with haptic experience by delivering realistic fashion material images and attaining important results. It also collected data for follow-up studies through validation via a user evaluation model. These data can be applied in various areas in addition to fashion materials, and follow-up studies can be provided with numerous suggestions with the data.
