Highlight
Background
This project was conducted through an industry-academia collaboration to develop a VR STEM education service for chemical labs. The initiative focused on creating a fundamental VR interaction tool to enhance learning experiences, starting with the micropipette, a key instrument in chemistry experiments.
Process
Co-design process of hardware and software
I independently managed the co-design process of hardware and software, from research to implementation. This end-to-end approach involved harmonizing physical and digital product design, prioritizing the integration of hardware and software to deliver an intuitive and realistic user experience.
In-depth Interview
User Scenario
VR Scene Design
Disassembly
HW Analysis
Sensor Selection
PCB Design
Research
Redefining VR experiment training for better skill development
Preliminary Research
Current VR training solutions rely heavily on standard controllers that lack the fidelity needed for skill-based training. I discovered that this limitation reduces the effectiveness of learning in VR, highlighting the needs for specialized haptic devices.
In-depth Interview
To fully understand the functional requirements of a micropipette, I conducted interview with two graduate-level teaching assistants in chemistry lab course. I gained essential ideas and process of using micropipette, as well as frequent mistakes that the students make like incorrect pipette tilt.
Design Opportunity
With a realistic haptic device, VR offers a practical platform for students to train with a micropipette.
By integrating VR with a haptic device, the tactile feedback of using a micropipette is replicated, creating a bridge between virtual training and physical lab practice. This method enables students to safely refine essential lab skills, enhancing their confidence and competence before handling actual equipment.
Project Scope
Comprehensive micropipette training model with core features
To ensure scalability, I developed a fully functional micropipette model as a foundational tool for chemistry labs. The tutorial scenario incorporates key features like plunger stops, volume adjustment, and tip ejection, providing a structured framework for core skill training that can expand to future experiments.
Key Features
The VR pipetting system replicates a real micropipette with functional plunger stops (first/second), volume adjustment, and tip ejection with tilt detection. These features provide realistic physical feedback, enabling immersive training that builds muscle memory and proper handling skills.
First Stop
Second Stop
Knob Rotation
Tip Eject
Tilt
Scenario
The VR pipetting system guides users through each step of the pipetting process, simulating real-world actions for a hands-on learning experience. From picking up the pipette to ejecting the tip, each stage is designed to replicate the tactile and functional aspects of using an actual micropipette, ensuring accurate and confident handling.
1) Grab the pipette
2) Attach a tip
3) Adjust the volume
4) First stop & release
5) Second stop & release
6) Eject the tip
Hardware Design
Physical device connecting virtual-physical world with tangible feedback
Creating a haptic-enabled pipette controller required precise coordination of mechanical and electrical design. This effort resulted in a seamless and highly responsive user experience, elevating the realism and functionality of virtual pipetting.
Iterative Mechanical Prototype
Through repeated prototyping and 3D modeling, I developed a form factor that closely mirrors a real micropipette in shape and feel. The design also accommodates PCB and sensor integration to ensure that all components work seamlessly together while performing essential functions in VR.
Disassembly
First Draft
Iteration
Final Prototype
Circuit Design
The custom-designed PCB integrates sensors that detect the user’s actions, such as plunger stops and volume adjustments. By carefully arranging components like the Hall effect sensor and rotary encoder, the circuit design ensures that every action within VR mirrors the tactile response of an actual micropipette.
Hardware Design
The final assembly brought together the casing, sensors, and internal components in a streamlined, ergonomic design. This hardware setup allows for wireless operation, enabling users to interact naturally within the VR environment without physical constraints, making the learning experience both accessible and realistic.
VR Scene Design
VR scene enhancing the physical with immersive feedback
The VR scene is designed to complement the physical pipette device by providing immersive visual and contextual feedback. Through intuitive cues and realistic interactions, the virtual environment bridges the gap between physical actions and digital simulations, enhancing the user’s overall training experience.
Playground
The virtual playground is equipped with all the necessary lab apparatus required for the simulation, creating a realistic and ready-to-use environment for effective training.
Guide Panel
At each step of the training, a display positioned directly in front of the user shows the current phase of the process, accompanied by supporting images and text-based guidance to enhance understanding.
Pipette Action Guide Tooltip
A tooltip above the pipette indicates the current action and provides instructions if the user deviates from the intended guidance. If the pipette is tilted excessively, it changes to yellow as a cautionary signal.
Action
Out of guidance
Tilt
High Fidelity Prototype
Integration of Hardware and Software for Immersive Training
The high-fidelity prototype is the result of HW/SW co-design process, integrating a haptic-enabled pipette controller with an immersive VR environment. This system combines realistic tactile feedback with interactive virtual guidance, delivering an intuitive and practical training solution that bridges the gap between physical and virtual lab experiences.