Sonomyography
Most upper limb amputees abandon their prosthesis because the control is not intuitive. Surface electromyography, the standard interface, suffers from a low signal-to-noise ratio, poor amplitude resolution, electrode displacement, sweat and fatigue. Sonomyography reads the muscle itself: ultrasound imaging of the forearm captures deep and superficial muscle deformation in detail, and this project turns that image stream into a real-time human-machine interface for prosthetic arm control.
The Hand Is Willing, the Electrode Is Weak
Prosthetic rejection is an interface problem before it is a mechanics problem.
From electrical surface potentials to imaged muscle motion
Losing an upper limb sharply reduces independence, and despite substantial advances in prosthetic technology, rejection rates for sophisticated devices remain high. The majority of upper limb amputees discard their prosthetics because of their complexity and the lack of intuitive control.
Surface electromyography, the most widely used control method, reads electrical activity through dry surface electrodes. It brings a low signal-to-noise ratio, poor amplitude resolution, an inability to capture complex muscle activations, and very poor sensitivity to finer muscle movement, on top of practical issues such as electrode displacement, sweat and fatigue.
Ultrasound imaging instead produces a detailed, dynamic mapping of the muscles themselves, with greater SNR and less hardware than sEMG, and machine learning on those images can associate muscle patterns with specific finger movements. Real-time prosthetic control from sonomyography had been demonstrated on custom wearable systems; what was missing is an end-to-end research platform where new control strategies can be prototyped on a full research scanner.
- ProblemHigh prosthetic rejection driven by non-intuitive control
- sEMG limitsLow SNR, poor amplitude resolution, complex activations missed, electrode displacement, sweat, fatigue
- AlternativeUltrasound imaging of deep and superficial forearm muscle
- GesturesPower grip, wrist pronation, point, rest state
From 500 fps Plane Waves to a 33 fps Decision
A research scanner, a screen grabber and a correlation matcher form a complete real-time control loop.
Every stage of the loop runs faster than the hand
Images are acquired on a Verasonics Vantage 128 channel research ultrasound platform using multi-angle plane wave transmission at more than 500 frames per second. The L11-5v probe sits in a custom 3D-printed holder strapped to the volar forearm, positioned 4 to 5 cm from the elbow joint so that both the deep and superficial flexor compartments fall in the scan plane.
The beamformed display is captured with the Python MSS screen grabber at 180 fps, downsized and cropped to the 480 by 480 pixel ultrasound region. Each live frame is compared, pixel by pixel, against stored representative images of each gesture, and the correlation feeds a k-nearest neighbours classifier. Matching sustains over 39 fps and classification over 33 fps, so the decision rate is set by the classifier, not the scanner.
During training, participants held each gesture for around ten seconds while the best scans were averaged into a single labelled representative image per gesture. During validation they performed random motions against a live scan.
- AcquisitionVerasonics Vantage 128, L11-5v probe, multi-angle plane waves, >500 fps
- CapturePython MSS screen grabber, 180 fps, cropped to 480×480 px
- MatchingPixel-by-pixel correlation against gesture representative images, >39 fps
- Classifierk-nearest neighbours on correlation scores, >33 fps
- Probe positionVolar forearm, 4–5 cm from the elbow, custom 3D-printed holder
Fourteen Joints Answer the Forearm
A ROS simulation closes the loop: the participant moves, and a robotic hand moves with them 0.6 seconds later.
Above 90% accuracy on four gestures, in real time
A 14 degree of freedom CAD model of a robotic arm that mimics a human hand was exported to the Unified Robotics Description Format and loaded into Rviz. Joint angles for power grip, wrist pronation, rest and point are assigned per gesture, the Verasonics host talks to the simulation laptop over TCP/IP, and the virtual hand starts moving as soon as the classifier fires, with a measured latency of 0.6 seconds.
Three healthy volunteers took part under ethics approval from IIT Palakkad, performing gesture repetitions first with enforced rest states between transitions, then switching gestures at random. Overall classification accuracy exceeded 90%: wrist pronation and rest were predicted perfectly, while point was occasionally projected as power grip.
Treating rest as its own motion class turned out to cost accuracy; removing it from cross-validation increased the motion discriminability for every participant. Future work targets comparison against sEMG, more subjects, and extension to a fuller gesture vocabulary, en route to intuitive control for individuals with transradial amputation.
- Simulation14 DOF URDF hand in ROS Rviz, TCP/IP link to the Verasonics host
- Latency0.6 s from predicted motion to virtual hand movement
- Subjects3 healthy volunteers, IIT Palakkad ethics approval, written informed consent
- Accuracy>90% over four gestures; wrist pronation and rest at 100%
- ConfusionPoint occasionally predicted as power grip; dropping rest raised discriminability
Watch the System Run
The full loop on video: gestures held on the bench, ultrasound frames classified live, and the ROS hand following along.
Video demonstration of the prototype: live forearm ultrasound driving the 14 DOF virtual hand in ROS.
Explore the Imaging Behind It
Sonomyography rides on the same high frame rate plane wave imaging the lab builds elsewhere: ultrafast acquisition, learned beamforming, and portable front ends.