Tsinghua Future Lab
/
Academic Project
Research findings were accepted to the 2025 ACM CHI Conference (as first author).
At Tsinghua Future Lab, we developed a tactile display for the visually impaired, a project supported by the China Disabled Persons' Federation. I also led experiments with blind schools, focusing on the application of tactile displays.
Tactile display for mass production, tactile image guidelines for education, audio description framework, data comics interaction method, and experiments with 16 participants.
May. 2024 - Sep.2024
Beijing.China
Existing challenges in blind education :
Limited Inclusion of the Blind in the Education System
One of the Biggest Barriers: Information Acquisition, Especially for Images
Advancements in technology have improved text access for the blind community. Visually impaired students in special education schools in China commonly use screen reading software and Braille displays. However, when images become a primary means of information dissemination, this poses significant barriers to their understanding.
Challenges in Image-Based Learning for Core Subjects
Visually impaired students, like their sighted peers, also learn foundational subjects such as math, physics, and Chinese, where images are essential. Due to display limitations, most illustrations in textbooks are either removed or simplified.
Example: Removal of Geometric Figures in Mathematical Proofs
Sighted students' mathematics textbooks
Blind students' mathematics textbooks
Traditional Methods for Image Display for the Blind
Falling Short in Material Variety
Thermoplastic
Modable but lacks the durability needed for long-term use.
Microsphere paper
Produces raised tactile graphics but is vulnerable to heat and light.
Dot printing
Requires costly materials for each use, limiting accessibility.
These methods fail to meet the demands of blind education for diverse, and reuseble learning materials.
Refreshing Tactile Display
Generating Unlimited Images Quickly and Efficiently
The average cost of tactile displays exceeds $15,000 USD, making them inaccessible to most users.
Market Analysis
The market lacks an affordable, scalable solution to meet the visual information needs of blind individuals.
Graillie Tactile Display
Graillie Tactile Display from Tsinghua Future Lab.
Raised dot patterns for touch-based images.
Two standout features
01 Much more affordable to make it more accessible.
Based on our cost analysis, the initial batch of 10 tactile displays will cost $7,000 each, 50% less than similar products. The price will drop to approximately $3,500 each for orders exceeding 1,000 units.
02: Patented Technology — Flip Latch Technology for Each Unit
The reduced device cost stems from using flip-latch technology, which balances refresh time, energy consumption, and cost. This design lowers material costs compared to piezoelectric devices, and the latest prototype actuates 3,600 pins in 0.8 seconds for content display.
Note: The entire team of students did not participate in the device production. The device was developed in collaboration with Chinese factories, and the materials used in the tactile display presentation were blended and designed by me.
Experiment 1 —— How to Better Transfer Sighted Images into Tactile Images and Enhance Understanding with Clearer Audio Descriptions.
01: Images Part — Investigating the Haptic Patterns and Preferences of Blind Individuals
Redesigned complex sighted images for tactile use, focusing on textbook illustrations in education.
02 Audio Part: Clearer and More Helpful Audio Description Framework
Tested various audio description methods to combine tactile images with structured audio guidance.
Experiment Goal
Experiment Recording
We conducted multiple experiments with blind students at the School of Special Education, Beijing Union University.
Image Experiment Results: Specific details and preferences for tactile images in the educational field.
We designed experiments with 12 sets of images, each drawn using different methods for the same content. Participants explored the images and selected the clearest ones, allowing us to summarize their preferences.
Audio Experiment Results: Optimal Audio Description Framework
This method can minimize their panic when they struggle to comprehend the tactile images.
Experiment 2 ——
Exploring a New Information Presentation Method : Tactile Data Comics
The Example of Tactile Data Comics — Presenting Information Step-by-Step
Data comics break down complex concepts into sequential tactile images, offering a step-by-step presentation method.
Experiment Design: Compare 3 Learning Conditions
We aim to explore whether tactile data comics can enhance information acquisition efficiency.
Experiment Materials: Examples from Math, Physics, Maps, and Literature Categories
Experiment Participant: 16 blind students from Beijing Union University participated.
While some participants had prior experience with touch-sensitive interfaces, none used or owned tactile displays regularly.
Experiment Procedure
Experiment Results Analysis
Participants' response accuracy analysis:
We employed a generalized linear mixed model with a logit link function, where the presentation method served as the main effect. We considered the subjects of the materials and types of blindness as random effects.
Main Effects: 3 Learning Conditions
Data comics resulted in higher response accuracy compared to static tactile graphics or audio narration.
Side Effects: 4 different subjects
Tactile data comics were more effective for certain subjects, suggesting their suitability varies.
Side Effects: Congenital vs. Acquired
Individuals with congenital or early blindness showed greater benefit from tactile data comics, but further studies are needed.
UEQ-S results analysis
Distribution of UEQ scores across 3 conditions
Participants rated tactile data comics as easier and more engaging compared to other methods.
Visual inspection of hand positions
Touch Motion Analysis
Heat maps revealed tactile data comics improved touch accuracy, aligning better with the image.
Qualitative analysis of interview responses
We transcribed the semi-structured interviews and used MaxQDA for inductive and deductive coding, focusing on participant actions relevant to our research. After resolving coding discrepancies, we grouped related codes using an affinity diagram to summarize themes.
Future Directions:
The Chinese government plans to adopt it widely for public schools, libraries, and the visually impaired, enabling more blind individuals to afford and use the device for learning and exploration.
Extended Research Directions:
Interaction Methods: Explore more effective ways to interact with tactile displays given their physical limitations.
Scalable Tactile Comics: Develop tools to automate the creation of tactile comics for diverse content.
Diverse Age Groups: Study tactile learning methods across different age groups and applications.
Special Thanks to :
Team Director: Prof. Jiao,
Assistant Professor, Tsinghua Future Lab
China Disabled Persons' Federation
for sponsoring mass production.
Rongluo for contributing to the data charts;
Xiwen for conducting experiments together.
Yuewen for analyzing experiments' results like
motion analysis, and creating heat maps.