NeuralBeamform
Adaptive beamformers such as minimum variance produce sharper, higher-contrast ultrasound images than delay and sum, and clinics almost never use them: the computation is far too heavy for real-time frame rates. This project trains neural networks to reproduce adaptive beamforming in a single forward pass, from CNNs that learn minimum variance from channel data, through models that learn during acquisition, to a tunable multibeamformer that blends three beamformers at once on edge hardware.
A CNN That Beamforms Like Minimum Variance
The starting point: train a convolutional network on real channel data to produce the image an adaptive beamformer would, without the adaptive cost.
Spatial context is the shortcut
The network is trained on time of flight corrected in-vivo channel data from plane wave transmits, with the corresponding minimum variance distortionless response (MVDR) image as the target. Where the earlier fully connected approach beamforms pixel by pixel, a CNN sees each pixel in its neighbourhood, making the learned apodization region adaptive during beamforming.
On the PICMUS benchmark the CNN improves on the fully connected baseline, and on MVDR itself, in both resolution and contrast, with a 6 dB CNR improvement using only the zero-angle transmission. The CNN matches the CNR that the baseline needs three times as many insonification angles to reach, and with about one-fifth of the total angles it beats both the baseline and DAS, opening the door to higher frame rates from fewer transmits.
- Training data600 in-vivo scans of arm and finger, Verasonics Vantage 128, L11-5v, 7.6 MHz
- TargetMVDR beamformed image from ToFC channel data
- BenchmarkPICMUS, 75 insonification angles from −16° to +16°
- Key result+6 dB CNR over the FCNN baseline at zero-angle transmission
The Sonographer Becomes the Ground Truth
Static models bet on one beamformer being best for every scan. Active learning lets the user decide, and trains the network while the probe is still on the skin.
Train during the scan, not after it
Neural beamformers are usually trained to mimic a single adaptive method, chosen in advance, after gigabytes of data have been collected and stored. This work describes, for the first time in this context, an active learning framework for beamforming: each acquisition is beamformed by DAS, F-DMAS, MVDR and GCF, the images are randomized in a GUI, and the one the user judges best becomes the ground truth for that batch.
A lightweight multi-scale U-Net trains concurrently with data acquisition on an NVIDIA Clara AGX Developer Kit, taking on average 0.5 seconds per training iteration. To keep selections consistent across users, the choice follows fixed criteria: axial and lateral resolution in high-intensity regions, speckle resolution in homogeneous regions, and contrast at cyst boundaries.
- CandidatesDAS, F-DMAS, MVDR and GCF beamformed images
- ModelLightweight U-Net based multi-scale network
- Edge hardwareNVIDIA Clara AGX Developer Kit beside a Verasonics research platform
- Iteration time0.5 s on average per training iteration, concurrent with acquisition
TGC-Net: Gain Compensation Without the Knobs
A beamformed image is only as readable as its gain profile. Time gain compensation is still adjusted by hand, so image quality depends on who is holding the probe.
Depth-aware gain, learned end to end
Sound attenuates as it travels, so echoes from deep tissue arrive weak and the operator compensates with anatomy-specific slider settings. TGC-net removes that dependence: a U-Net inspired model with a contracting and expansive path takes the uncompensated image and predicts the full gain matrix, optimizing lateral and axial gains with depth.
Trained on 2000 simulated and experimental datasets from a C5-2 curvilinear array, the model was validated with FWHM, CNR, generalized CNR and SNR against the algorithmic ground truth. Point targets resolve at 3.7 mm lateral and 3.6 mm axial FWHM, and on the simulated test image the prediction edges past the ground truth with 1.38% higher CNR and 1.1% higher gCNR on hyperechoic cysts.
- ModelU-Net inspired, AntiRectifier activation, 614×348 gain matrix output
- Training2000 datasets, simulated (SIMUS/MUST) and experimental, 50 epochs, PyTorch
- AcquisitionC5-2 curvilinear array, 128 elements, 3.125 MHz, Verasonics Vantage 128
- PhantomsFetal US Biometrics Model 068 and CIRS 054GS, 0.5 dB/(MHz·cm)
A Reinforcement Learning Agent Tunes the Beamformer
Needles reflect like mirrors and vanish from conventional images. Instead of learning an image, the network here learns a decision: which aperture to use.
The aperture becomes an action
In ultrasound guided interventions the needle’s specular echo is highly directive, and beamforming tuned for diffuse tissue suppresses it. Adaptive beamformers can recover it by choosing the right aperture, but that choice depends on the insertion at hand. Here a Deep Q-Network agent selects the sub-aperture length of a minimum variance beamformer: the beamforming process is the environment, each beamformed image is a state, and the reward is a weighted sum of contrast, resolution, speckle quality and the linearity of the needle.
The approach is unsupervised, needing no ground-truth images, and across in-vitro phantom insertions and ex-vivo beef tissue it shows a marked improvement in needle visibility over conventional beamforming while preserving the surrounding image.
- AgentDeep Q-Network choosing the MV sub-aperture length
- RewardWeighted contrast, resolution, speckle quality and needle linearity
- In-vitroPhantom insertions at 15°, 20° and 30°
- Ex-vivoBeef tissue with 15G and 21G needles
ADAPT: A Tunable Multibeamformer
Earlier neural beamformers mimic one beamformer at one depth and retrain for anything else. ADAPT learns three at once and hands the blend to the clinician.
Blend beamformers with three sliders
ADAPT (Adaptive Depth-Agnostic Patch-wise Tunable-multibeamformer) trains three identical U-Net based apodization networks, one each for DAS, FDMAS and MV, and fuses their learned weights by tunable weighted averaging, with the mix set by three coefficients that sum to one. One fused network then beamforms in a single pass, blending the acceptability of DAS, the contrast of FDMAS and the resolution of MV. Depth-wise patching makes the learning depth independent: no retraining for a new imaging depth.
The efficiency gains are large: 73% faster training, 94% fewer computations and 94% less activation memory than a model trained over the full depth. On PICMUS, lateral FWHM improves 41.4% over DAS in-silico, within 2.4% of MV itself, and in-silico gCNR improves 20.7% over FDMAS. Patch-wise learning also preserves 25–46% more signal beyond 15 mm depth, keeping image quality uniform across the imaging range.
- FusionTunable weighted averaging of DAS, FDMAS and MV model weights
- Training data1000 wrist acquisitions from healthy volunteers, L11-5v, 7.6 MHz
- Efficiency73% training speedup, 94% fewer computations, 94% less activation memory
- ResolutionLateral FWHM 41.4% better than DAS in-silico, within 2.4% of MV
- ImplementationPyTorch with CuPy-accelerated ToFC on an NVIDIA Tesla T4, 16 GB
Contributors and Partners
Five years of neural beamforming across IIT Palakkad and the Singapore Institute of Technology, with NVIDIA AI Technology Centre and Samsung R&D Institute India-Bangalore as industry partners.
Region adaptive CNN beamforming for plane wave imaging, the 2021 study that opened the programme.
Profile →Active learning framework for beamforming, training the network while the data is acquired.
Profile →Patch based transformation for approximating minimum variance images in a delay and sum pipeline.
Profile →Reinforcement learning augmented beamforming for needle tracking, bridging to the NeedleTrack project.
Profile →Co-author across the active learning and TGC-net studies, from beamforming to gain compensation.
Profile →TGC-net, the deep learning model that replaces manual time gain compensation, with Samsung R&D.
ADAPT, the depth-agnostic tunable multibeamformer, with NVIDIA AI Technology Centre, Singapore.
Principal investigator across all six works, IIT Palakkad and Singapore Institute of Technology.
Biography →Explore the Wider Programme
Neural beamforming is one arm of the Cognitive Ultrasound programme: the physics-driven counterpart, specular beamforming and needle tracking, has pages of its own.