DexHand connects two worlds that are often developed separately: perception in software and movement in hardware. A camera observes a hand, a vision pipeline extracts a gesture, and an embedded controller converts that interpretation into servo motion.
The demo looks simple when it works. The system behind it is a chain of components with different timing, failure modes, and coordinate systems.
The perception pipeline
The camera feed enters a computer-vision process built around OpenCV and hand landmarks. Raw pixels are not useful to the controller, so the first task is to produce a stable representation of finger positions and joint relationships.
Gesture recognition needs smoothing. A single noisy frame should not make a robotic finger jump. I use short temporal windows and thresholds so a movement must remain consistent before it becomes a command.
Communication and control
ROS2 provides the messaging layer between perception and control. Keeping these responsibilities separate matters. The vision node can publish a normalized hand state without knowing the exact servo arrangement. The controller can subscribe to that state and map it to the mechanical limits of the hand.
An ESP32 receives the target positions and drives the servos. Every joint needs calibration because the same pulse width does not produce the same safe motion across different assemblies. Limits protect the mechanism from commands that are valid in software but impossible in hardware.
What robotics changes
Physical systems make bugs visible. Latency becomes hesitation. Noise becomes vibration. A wrong sign becomes motion in the wrong direction. Good logs still matter, but observation and measurement matter just as much.
DexHand taught me to design interfaces between domains: vision to messages, messages to control, and control to mechanics. The most interesting engineering often happens at those boundaries, where no single library can solve the complete problem.