The rapid advancement of internet technologies has allowed us to communicate visual and auditory information with exceptional clarity and speed. Now, a groundbreaking evolution is on the horizon: the ability to transmit the sense of touch digitally. This development promises to revolutionize fields such as remote surgery, online gaming, and many other applications that require high-fidelity haptic feedback. This article delves deeply into the new “Haptic Codecs for the Tactile Internet” (HCTI) standard, which enables the efficient transmission of haptic information across networks.
Introduction to HCTI
The HCTI standard, detailed in a paper published on June 14 by the IEEE Standards Association, represents a significant milestone in haptic technology. Led by Professor Eckehard Steinbach from the Technical University of Munich (TUM), the research team has developed a codec that compresses haptic data to be transmitted at a rate close to the human perception threshold. This innovation reduces the data packet transmission rate from 4,000 times per second to just 100 times per second, significantly lowering network demands while maintaining realistic feedback.
“Haptic Codecs for the Tactile Internet” (HCTI) and their applications, technical details, and relevant updates.
| Aspect | Details |
|---|---|
| Definition | Haptic Codecs for the Tactile Internet (HCTI) are data compression standards that allow for the efficient transmission of haptic information (touch feedback) over the internet. |
| Purpose | To enable remote physical interaction with realistic touch experiences by compressing and transmitting haptic data efficiently over networks with minimal latency. |
| Key Features | – Compression of haptic data to reduce packet size and transmission rate – Optimized control loop between sender and receiver<br>- Similar to JPEG/MPEG for haptics |
| Primary Applications | – Remote surgery and teleoperation – E-Commerce with tactile feedback (T-Commerce) – Telepresence and communication (T-Skype) – Virtual Reality (VR) and gaming |
| Technological Innovations | – Reduction of data packet transmission rate from 4,000 to 100 times per second – Efficient compression of tactile and kinesthetic data – Development of kinesthetic and tactile codecs |
| Components of Haptic Data | – Kinesthetic Information: Forces, torques, position, and movement of body parts – Tactile Information: Texture, temperature, fine surface properties |
| Compression Techniques | – Autoregressive Moving Average (ARMA) modeling – Discrete Cosine Transform (DCT) compression – Feature extraction for parametric representations |
| Transmission Protocols | – Handshaking protocols for device capability exchange – Multiplexing schemes for joint transmission of video, audio, and haptic data |
| Quality Evaluation Measures | – Objective and subjective evaluation of haptic data quality – Use of psychophysics and human perception studies |
| Standardization Efforts | – IEEE P1918.1.1 standardization group – Approved hardware and software reference setup for tactile codec development |
| Challenges | – Achieving ultra-low latency and high bandwidth for haptic data transmission – Standardization and interoperability across diverse devices and applications – Designing intuitive and user-friendly haptic interfaces |
| Future Directions | – Integration with advanced VR and AR systems – Enhancing remote education and training with haptic feedback – Improving accessibility and assistive technologies for individuals with disabilities |
| Recent Updates | – Continued research and development in haptic feedback technologies – Updates on standardization progress and implementation in various industries – Increasing adoption of haptic feedback in consumer electronics and virtual environments |
| Potential Impact | – Revolutionizing remote surgery with precise haptic feedback – Transforming online shopping with the ability to feel products – Enhancing telepresence and communication with realistic touch experiences – Making VR and gaming more immersive |
Technological Background
Haptic Technology and Its Challenges
Haptic technology involves the sense of touch, including tactile sensations and kinesthetic interactions. While visual and auditory technologies have reached high levels of sophistication, haptic technology has lagged. In the context of the Tactile Internet, haptic solutions are essential for enabling remote physical interactions with high fidelity, allowing users to feel and interact with distant objects as if they were physically present.
Components of Haptic Data
Haptic data consists of two main submodalities: kinesthetic and tactile information. Kinesthetic information relates to the perception of forces, torques, and the position of body parts, while tactile information pertains to the texture, temperature, and fine surface properties of objects. The compression of kinesthetic information has been extensively studied, whereas the compression of tactile data is a newer area of research.
Current State and Innovations
Traditionally, transmitting tactile feedback over a remote connection requires data packets to be sent both ways 4,000 times per second. While this ensures realistic feedback and robust data transmission, it places high demands on the network. To overcome this, the HCTI standard uses compression techniques and reduces the transmission rate to 100 times per second, close to the human perception threshold. The HCTI standard optimizes the control loop between the sender and receiver, compressing information similarly to how audio or image files are sent over the internet, but in a two-way format.
Applications and Use Cases
Remote Surgery and Teleoperation
One of the primary applications of the HCTI standard is in remote surgery. By transmitting haptic feedback, surgeons can perform operations with the same precision as if they were physically present. This capability also extends to teleoperation scenarios, where robotic arms can be controlled from a distance, offering potential in fields such as space exploration and hazardous material handling.
In a typical setup, the user is connected to a kinesthetic input/output device, and the teleoperator is realized using a robotic arm equipped with force sensors, a video camera, a microphone, and an end-effector or tool. The HCTI standard’s ability to compress and efficiently transmit haptic data ensures that the feedback is realistic, allowing for precise control and manipulation.
E-Commerce with Tactile Feedback (T-Commerce)
Online shopping experiences can be significantly enhanced by enabling customers to feel the texture of products through their screens. This development opens up a new realm of T-Commerce, where the tactile experience is as crucial as the visual one. For example, novel tactile feedback mechanisms can be integrated into touch screens to simulate the fine surface roughness of products.
A high-fidelity T-Commerce scenario requires a comprehensive representation of the object, including all relevant kinesthetic and tactile properties. Ideally, the user will not be able to distinguish between touching the real object and the provided online experience. This level of immersion is made possible by the HCTI standard, which ensures that tactile information is accurately captured, transmitted, and reproduced.
Telepresence and Communication (T-Skype)
Adding touch to telepresence systems like video conferencing can enhance the feeling of presence and interaction. This could be particularly beneficial in scenarios such as remote emotional support, where a gentle touch can convey more than words. Current telepresence systems exchange high-quality audio and video but lack the capability to transmit touch experiences. The integration of tactile actuators and haptic feedback mechanisms can bridge this gap, enabling more immersive and emotionally resonant interactions.
Virtual Reality (VR) and Gaming
The integration of haptic feedback in VR systems can provide a more immersive experience. Users can interact with virtual objects and feel their textures, making the virtual world more tangible and engaging. While current VR systems primarily focus on visual and auditory information, the addition of haptic feedback can significantly enhance the realism and interactivity of virtual environments.
For instance, users in a virtual showroom can touch and feel material samples, receiving both kinesthetic and tactile feedback. This capability is essential for applications such as virtual shopping, training simulations, and interactive entertainment. The HCTI standard ensures that haptic information is transmitted efficiently and accurately, enabling high-fidelity tactile interactions in virtual environments.
Technical Contributions and Standards
The paper outlines several technical contributions that pave the way for the development and standardization of haptic codecs:
- Kinesthetic Codec Development: The research includes the development of a kinesthetic codec under the IEEE P1918.1.1 standardization group. This codec has demonstrated remarkable data reduction performance through extensive cross-validation experiments.
- Tactile Processing Pipeline: A novel tactile processing pipeline has been introduced, covering the acquisition of surface material properties, processing sensor signals, compressing tactile data, and presenting tactile experiences to the user. The pipeline ensures that tactile information is captured accurately, processed efficiently, and reproduced with high fidelity.
- Hardware and Software Reference Setup: The paper presents the approved reference setup for tactile codec development, including a sensorized surface material scanning tool and a voicecoil-based display. This setup serves as a benchmark for evaluating the performance and effectiveness of different tactile codecs.
- Quality Evaluation Measures: The research provides an overview of objective quality evaluation measures for kinesthetic information, validated through both objective and subjective experiments. These measures ensure that the transmitted haptic information maintains high quality and provides a realistic touch experience.
Detailed Analysis of Haptic Data Compression
Kinesthetic Information
Kinesthetic information includes data related to the position, velocity, and forces experienced by body parts during interactions with objects. The sensory information provided by mechanoreceptors in muscles, tendons, and joints is collectively referred to as the kinesthetic sense. This information contributes to the perception of limb position, movement, and applied forces, helping determine the physical properties of touched objects, such as viscosity, stiffness, and inertia.
To transmit kinesthetic information efficiently, data reduction schemes based on perceptual deadbands have been developed. These schemes leverage Weber’s law of just-noticeable differences, which states that only if the relative difference between two subsequent stimuli exceeds a certain threshold will the signal be perceivable and need to be transmitted. By applying this principle, the data packet rate can be significantly reduced without compromising the quality of the haptic feedback.
Tactile Information
Tactile information pertains to the perception of fine surface properties, such as texture and temperature. Human tactile perception relies on mechanoreceptors in the skin, which respond to different types of stimuli. For example, Meissner corpuscles are responsible for perceiving fine roughness, while Pacinian corpuscles detect high-frequency vibrations.
Capturing relevant tactile information involves measuring normal and tangential interaction forces, surface temperature, and high-frequency vibrations. These data points are then processed and compressed to reduce the data size while preserving the essential tactile features. Various techniques, such as autoregressive moving average (ARMA) modeling and discrete cosine transform (DCT) compression, are employed to achieve efficient tactile data compression.
Advanced Topics in Haptic Communication
Multiplexing and Handshaking Protocols
Multiplexing schemes support the simultaneous transmission of multiple data streams over the same communication channel. In the context of the Tactile Internet, multiplexing refers to the joint transmission of video, audio, and haptic data streams. The multiplexing scheme allocates appropriate network resources to each stream based on application needs and available bandwidth, ensuring synchronized delivery of multi-sensory information.
Handshaking protocols for haptic devices facilitate the exchange of device capabilities, such as the number of degrees of freedom and maximum input/output values. These protocols ensure compatibility and optimal performance across different haptic devices and applications.
Tactile Internet Meta Data (TIM)
As haptic devices become widely deployed, a standard technology-neutral metadata format, known as Tactile Internet Metadata (TIM), is necessary. TIM provides a comprehensive description of haptic application components, including device specifications, haptic rendering mechanisms, and quality of experience parameters. This standardized metadata format ensures interoperability and facilitates the development of haptic applications across diverse platforms.
Integration with Stability-Ensuring Control Schemes
Stability-ensuring control schemes, such as wave-variable transformation and time-domain passivity approach, are essential for maintaining stable closed-loop kinesthetic communication in the presence of communication delays. These control schemes are integrated with kinesthetic data reduction algorithms to ensure both stability and efficient data transmission.
The Future of the Tactile Internet
The Tactile Internet represents a shift from content-oriented to control-oriented communication, enabling human skills to be delivered remotely. This paradigm shift requires the exchange of multi-sensory information, particularly haptic data, to provide remote physical experiences globally. The development of the HCTI standard marks a significant milestone in this evolution, paving the way for a more immersive and interactive digital future.
Potential Applications and Innovations
The potential applications of the Tactile Internet are vast and varied. In addition to remote surgery and e-commerce, other promising areas include:
- Education and Training: Haptic feedback can enhance remote learning experiences by allowing students to interact with virtual models and simulations. Medical students, for example, can practice surgical techniques on virtual patients, receiving realistic tactile feedback.
- Manufacturing and Quality Control: Haptic technology can improve remote inspection and quality control processes in manufacturing. Inspectors can feel the texture and properties of materials, ensuring high standards are maintained even from a distance.
- Accessibility and Assistive Technologies: Haptic feedback can aid individuals with visual impairments by providing tactile information about their environment. This technology can enhance navigation aids and improve the usability of digital devices for people with disabilities.
Challenges and Future Directions
Despite the significant advancements, several challenges remain in the development and deployment of the Tactile Internet. These challenges include:
- Latency and Bandwidth: Achieving ultra-low latency and high bandwidth for haptic data transmission is crucial for providing realistic touch experiences. Ongoing research focuses on optimizing network protocols and infrastructure to meet these requirements.
- Standardization and Interoperability: Ensuring compatibility across diverse haptic devices and applications requires standardized protocols and metadata formats. Continued efforts in standardization will facilitate widespread adoption and integration of haptic technology.
- User Experience and Adoption: Understanding user preferences and designing intuitive haptic interfaces are essential for widespread adoption. User studies and feedback will play a crucial role in refining haptic technologies and ensuring they meet user needs.
The development of the HCTI standard marks a significant milestone in the evolution of the internet. By enabling the transmission of touch, this technology has the potential to revolutionize various fields, from remote surgery to online gaming. The ongoing standardization efforts and technical advancements in haptic codecs are paving the way for a more immersive and interactive digital future.
The complete details and contributions of this research are available in the paper published by the IEEE Standards Association. This paper provides a comprehensive overview of the state-of-the-art in haptic technology, the development of haptic codecs, and the potential applications and future directions for the Tactile Internet. As the technology continues to evolve, it will undoubtedly unlock new possibilities and transform the way we interact with digital and remote environments.
