Highlight
Background
This project was conducted through an industry-academia collaboration to develop a VR STEM education service for chemical labs. The initiative focused on creating a fundamental VR interaction tool to enhance learning experiences, starting with the micropipette, a key instrument in chemistry experiments.
Process
As a individual project, I led the co-design of hardware and software, driving simultaneous development for seamless integration.
Research
Design & Develop
Validate
Desk Research
Expert Interview
Hardware 🅗
Mechanical Design
Circuit Design
Signal Processing
Hardware Design
Software 🅢
User Scenario
VR Scene Design
VR Interaction
VR Develop
User Testing
Client Feedback
Background Research
🚫 Limited Access to Resources
Limited access to laboratory time and insufficient equipment made it challenging for students to conduct experiments effectively.
Reference
JH Lee, & HG Hong (2022). A Case Study on the Difficulties of Experiments in the Process of Open Inquiry on Chemistry Experienced by Graduates of Science-Gifted Education Institute
🤚 Absence of physical fidelity
The lack of physical fidelity between VR controllers and actual pipettes is a key factor of high dexterity errors in VR-trained students.
Reference
Petersen, G. B., Klingenberg, S., & Makransky, G. (2022). Pipetting in Virtual Reality Can Predict Real-Life Pipetting Performance
How might we
How might we replicate micropipette in VR
for a realistic learning experience?
Project Scope
Micropipette guide for beginners with full feature simulation
This project focused on implementing foundational micropipette functions by designing and prototyping a tutorial process for beginner training.
Expert Interview
Functional Requirements
Based on expert interviews, I finalized four essential physical functions (first/second stop, knob rotation, tip eject) and a mistake prevention feature (tilt detection).
Essential physical functions
First Stop
Second Stop
Knob Rotation
Tip Eject
Mistake Prevention
Tilt Detection
Tutorial Scenario
1) Grab the pipette
2) Attach a tip
3) Adjust the volume
4) First stop & release
5) Second stop & release
6) Eject the tip
Hardware Design
Physical device connecting virtual-physical world with tangible feedback
I managed the integration of mechanical and electrical systems to build a tangible pipette controller.
Iterative Mechanical Prototype
Through iterative prototyping and 3D modeling, I developed a realistic micropipette form factor, integrating PCB and sensors
Disassembly
Disassembling the micropipette to understand the mechanical structure for each functional requirements.
First Draft
Focusing on the mechanical structure as the first draft
Iteration
After PCB Design, structure integrated with the boards
Final Prototype
Integration of mechanical and electrical components completed
Functional Requirement to Sensor Selection
Based on the findings gained from expert interview, I firstly
1
Hall Effect Sensor
The sensor detects the states of both the plunger and the tip ejector by sensing a magnet. It is also capable of differentiating between the first and second stop positions.
2
Rotary Encoder
This detects the rotation of the volume adjusting knob and accordingly changes the volume setting.
3
IMU Sensor
When the tilt level of the device along the y-axis reaches a certain threshold, it triggers an alarm.
4
Battery Charging Module
This device operates wirelessly with its equipped rechargeable battery.
Circuit Design
The custom PCB integrates sensors to detect actions like plunger stops and volume adjustments, ensuring VR interactions closely mimic a real micropipette.
VR Scene Design
VR scene enhancing the physical with visual feedback
The VR scene is designed to complement the physical pipette device by providing immersive visual and contextual feedback. Through intuitive cues and realistic interactions, the virtual environment bridges the gap between physical actions and digital simulations, enhancing the user’s overall training experience.
Left and right hand interaction
The left hand operates the panel interface, while the right hand is dedicated to precise micropipette handling, ensuring seamless multitasking in the VR environment.
Guide panel updates automatically with the process.
At each step of the training, a display positioned directly in front of the user shows the current phase of the process, accompanied by supporting images and text-based guidance to enhance understanding.
Tooltip above the pipette displays the current status and guides users.
The tooltip is positioned above the pipette, considering the user’s line of sight. Interactions are designed to vary based on three scenarios: following the guide, deviating from the guide, and tilting the pipette.
Correct action
Off-guide action
Tilt
High Fidelity Prototype
Integration of Hardware and Software for Immersive Training
The high-fidelity prototype, developed through HW/SW co-design, integrates a haptic pipette controller with an immersive VR environment, offering realistic tactile feedback and interactive virtual guidance for effective training.
Reflection
Bridging Physical and Virtual through HW/SW Co-design
I focused on integrating hardware and software to create a seamless, immersive training experience. By balancing physical fidelity with practical usability, I learned the importance of iterative prototyping and real-time feedback in refining both design and functionality. This process highlighted the value of rapid iteration and cross-disciplinary problem-solving.