Publications

Three-dimensional (3D) freehand ultrasound (US) is a widely used imaging modality that allows non-invasive imaging of medical anatomy without radiation exposure. Surface reconstruction of US volume is vital to acquire the accurate anatomical structures needed for modeling, registration, and visualization. However, traditional methods cannot produce a high-quality surface due to image noise. Despite improvements in smoothness, continuity, and resolution from deep learning approaches, research on surface reconstruction in freehand 3D US is still limited. This study introduces FUNSR, a self-supervised neural implicit surface reconstruction method to learn signed distance functions (SDFs) from US volumes. In particular, FUNSR iteratively learns the SDFs by moving the 3D queries sampled around volumetric point clouds to approximate the surface, guided by two novel geometric constraints: sign consistency constraint and on-surface constraint with adversarial learning. Our approach has been thoroughly evaluated across four datasets to demonstrate its adaptability to various anatomical structures, including a hip phantom dataset, two vascular datasets and one publicly available prostate dataset. We also show that smooth and continuous representations greatly enhance the visual appearance of US data. Furthermore, we highlight the potential of our method to improve segmentation performance, and its robustness to noise distribution and motion perturbation.

A 3-D ultrasound (US) imaging technique has been studied to facilitate the diagnosis of spinal deformity without radiation. The objective of this article is to propose an assessment framework to automatically estimate spinal deformity in US spine images. The proposed framework comprises four major components, a US spine image generator, a novel transformer-based lightweight spine detector network, an angle evaluator, and a 3-D modeler. The principal component analysis (PCA) and discriminative scale space tracking (DSST) method are first adopted to generate the US spine images. The proposed detector is equipped with a redundancy queries removal (RQR) module and a regularization item to realize accurate and unique detection of spine images. Two clinical datasets, a total of 273 images from adolescents with idiopathic scoliosis, are used for the investigation of the proposed framework. The curvature is estimated by the angle evaluator, and the 3-D mesh model is established by the parametric modeling technique. The accuracy rate (AR) of the proposed detector can be achieved at 99.5%, with a minimal redundancy rate (RR) of 1.5%. The correlations between automatic curve measurements on US spine images from two datasets and manual measurements on radiographs are 0.91 and 0.88, respectively. The mean absolute difference (MAD) and standard deviation (SD) are 2.72\^}circ }pm 2.14\^}circ and 2.91\^}circ }pm 2.36\^}circ , respectively. The results demonstrate the effectiveness of the proposed framework to advance the application of the 3-D US imaging technique in clinical practice for scoliosis mass screening and monitoring.

The objective of this study is to develop a deep-learning-based detection and diagnosis technique for carotid atherosclerosis (CA) using a portable freehand 3-D ultrasound (US) imaging system. A total of 127 3-D carotid artery scans were acquired using a portable 3-D US system, which consisted of a handheld US scanner and an electromagnetic (EM) tracking system. A U-Net segmentation network was first applied to extract the carotid artery on 2-D transverse frame, and then, a novel 3-D reconstruction algorithm using fast dot projection (FDP) method with position regularization was proposed to reconstruct the carotid artery volume. Furthermore, a convolutional neural network (CNN) was used to classify healthy and diseased cases qualitatively. Three-dimensional volume analysis methods, including longitudinal image acquisition and stenosis grade measurement, were developed to obtain the clinical metrics quantitatively. The proposed system achieved a sensitivity of 0.71, a specificity of 0.85, and an accuracy of 0.80 for diagnosis of CA. The automatically measured stenosis grade illustrated a good correlation ( r = 0.76) with the experienced expert measurement. The developed technique based on 3-D US imaging can be applied to the automatic diagnosis of CA. The proposed deep-learning-based technique was specially designed for a portable 3-D freehand US system, which can provide a more convenient CA examination and decrease the dependence on the clinician’s experience.

As an emerging medical diagnostic technology, photoacoustic imaging has been implemented for both preclinical and clinical applications. For clinical convenience, a handheld free-scan photoacoustic tomography (PAT) system providing 3D imaging capability is essentially needed, which has potential for surgical navigation and disease diagnosis. In this paper, we proposed a free-scan 3D PAT (fsPAT) system based on a handheld linear-array ultrasound probe. A global positioning system (GPS) is applied for ultrasound probe’s coordinate acquisition. The proposed fsPAT can simultaneously realize real-time 2D imaging and large field-of-view 3D volumetric imaging, which is reconstructed from the multiple 2D images with coordinate information acquired by the GPS. To form a high-quality 3D image, a dedicated space transformation method and a reconstruction algorithm are used and validated by the proposed system. Both simulation and experimental studies have been performed to prove the feasibility of the proposed fsPAT. To explore its clinical potential, in vivo 3D imaging of human wrist vessels is also conducted, showing a clear subcutaneous vessel network with high image contrast.

Osteopenia is considered a common phenomenon in patients who have scoliosis. Quantitative ultrasound has been used to assess skeletal status for decades, and recently ultrasound imaging using reflection signals from vertebrae were as well applied to measure spinal curvatures in children with scoliosis. The objectives of this study were to develop a new method that can robustly extract a parameter from ultrasound spinal data for estimating bone quality of scoliotic patients and to investigate the potential of the parameter in predicting curve progression. The frequency amplitude index (FAI) was calculated based on the spectrum of the original radiofrequency signals reflected from the tissue-vertebra interface. The correlation between FAI and reflection coefficient was validated using decalcified bovine bone samples in vitro, and the FAIs of scoliotic subjects were investigated in vivo with reference to body mass index, Cobb angles and curve progression status. The results revealed that the intra-rater measures were highly reliable between different trials (intra-class correlation coefficient = 0.997). The FAI value was strongly correlated with the reflection coefficient of bone tissue (R2 = 0.824), and the lower FAI indicated the higher risk of curve progression for the non-mild scoliosis cases. This preliminary study found that the FAI method can provide a feasible and robust approach to assessment of the bone quality of spine and may be a promising factor in monitoring curve progression of patients who have adolescent idiopathic scoliosis.

Three-dimensional (3-D) freehand ultrasound (US) imaging has been applied to the investigation of spine deformity. However, it is a challenge for the current 3-D imaging reconstruction algorithms to achieve a balance between image quality and computation time. The objectives of this article are to implement a new fast reconstruction algorithm that can fulfill the request of immediate demonstration and processing for high-quality 3-D spine imaging, and to evaluate the reliability and accuracy of scoliotic curvature measurement when using the algorithm. The fast dot-projection (FDP) algorithm was applied for voxel-based nearest neighbor (VNN), multiple plane interpolation (MPI), and pixel nearest neighbor (PNN) protocols to reduce the reconstruction time. The 3-D image volume was reconstructed from the datasets acquired from scoliotic subjects. The computational cost, image characteristics, and statistical analyses of curve measurements were compared and evaluated among different reconstruction protocols. The results illustrated that the 3-D spine images using the FDP-MPI4 algorithm showed higher brightness (20%), contrast (14%), and signal-to-noise ratio (SNR) (26%) than FDP-VNN. The measurement performed by trainee rater exhibited significant improvement in measurement reliability and accuracy using FDP-MPI4 in comparison with FDP-VNN ( \p < 0.01 ), and the intraclass correlation coefficient (ICC) of interrater measurement increased from 0.88 to 0.96. The FDP-PNN method could acquire and reconstruct spine images simultaneously and present the results in 1–2 min, which showed the potential to provide the approximate real-time visualization for fast screening.

The reconstruction of high-quality shape geometry is crucial for developing freehand 3D ultrasound imaging. However, the shape reconstruction of multi-view ultrasound data remains challenging due to the elevation distortion caused by thick transducer probes. In this paper, we present a novel learning-based framework RoCoSDF, which can effectively generate an implicit surface through continuous shape representations derived from row-column scanned datasets. In RoCoSDF, we encode the datasets from different views into the corresponding neural signed distance function (SDF) and then operate all SDFs in a normalized 3D space to restore the actual surface contour. Without requiring pre-training on large-scale ground truth shapes, our approach can synthesize a smooth and continuous signed distance field from multi-view SDFs to implicitly represent the actual geometry. Furthermore, two regularizers are introduced to facilitate shape refinement by constraining the SDF near the surface. The experiments on twelve shapes data acquired by two ultrasound transducer probes validate that RoCoSDF can effectively reconstruct accurate geometric shapes from multi-view ultrasound data, which outperforms current reconstruction methods. Code is available at https://github.com/chenhbo/RoCoSDF.

Two-dimensional (2D) intraoral ultrasonography has shown promising potential to image the periodontal structures without the ionizing radiation. To advance the clinical application of intraoral ultrasonography, a novel ultrasound imaging system is necessary to image the structure of periodontal anatomy in 3D space. In this study, we developed a portable 3D intraoral imaging system by consisting of a handheld high-frequency linear intraoral ultrasound transducer (up to 23 MHz) and an optical tracking system. The proposed system can consistently provide the 2D images with 3D poses information for the real-time online reconstruction. After the online reconstruction, a total variation regularization (TVR) technique was adopted to filter the noise in raw tracking poses during the freehand scanning. The validation was performed on two incisors of a mandibular phantom, and the reconstructed volume from the proposed system was compared with the volume from Micro Computed Tomography (\textmuCT). The global average absolute and relative errors measured on the width and height of the two incisors were 0.16\textpm0.06 mm and 2.63\textpm1.50%, respectively. The TVR-corrected reconstructed volumes revealed a smoother reconstructed surface. This in-vitro study demonstrated a comparable result of the proposed system with \textmuCT to image the dental phantom.

Accurate 3D geometric shapes of carotid arteries are imperative for the three-dimensional (3D) ultrasound (US) imaging to clinically assess the carotid atherosclerosis (CA). However, the traditional surface reconstruction method, such as iso-surface (ISO-SURF), suffers from the image noise, voxel resolution and additional processing. In this paper, we introduce the neural implicit representation based on the deep learning network for the 3D surface reconstruction of media-adventitia boundary (MAB), plaque, and lumen-intima boundary (LIB) together. The unsigned distance functions are learned by the network to generate the mesh. For the validation, six volumes were simulated in carotid shape with MAB and LIB, and random noise was added around the boundaries. The results showed that experiments on six simulated volumes illustrated better performance than ISO-SURF, with 47%, 36%, and 55% decrease in Chamfer distance, average absolute distance, and Hausdorff distance, respectively. The visualization results from the CA clinical data revealed a smoother and more continuous geometric surface than ISO-SURF. The comparison result has shown the potential of the proposed method to examine vascular pathologies in the future.

Convolutional Neural Network (CNN) based method has been shown as a promising tool for automatic curve assessment of scoliosis without time-consuming manual measurement. However, CNN-based method was not reliable to accurately identify vertebral levels. The objective of this study is to investigate the performance of DEtection with TRansformer (DETR) for lamina detection, spine curve measurement and vertebral level identification. The implementation process included the following three steps: 1) automatic detection of lamina pairs based on deep learning methods; 2) assessment of the spinal curvature; 3) identifying the vertebral levels. Total 254 ultrasound images obtained from scoliotic patients were used for the training and evaluation of the proposed method. The average precision (AP) of lamina pairs detection using DETR and Faster R-CNN were 86.12 and 82.03, respectively. Compared to Faster R-CNN, DETR showed smaller MAD (4.47\textdegree\pm3.71\textdegree vs 5.3\textdegree\pm4.77\textdegree ) and higher correlation (0.90 vs 0.85), especially the MAD of DETR was less than the clinical acceptable error (5\textdegree ). For vertebral level identification, the mean accuracy rate using DETR was increased by 2.8% than Faster R-CNN, while the mean error rate and redundancy rate was reduced by 62% and 71%. The results demonstrated that the transformer-based detection network could achieve more reliable performance on scoliotic curve assessment and vertebral level identification.

Photoacoustic imaging (PA) is an emerging imaging technology with various modalities for 2D and 3D imaging in biomedical community. To promote the clinical application of this technology, a 3D photoacoustic tomography (PAT) system that can realize flexible scanning and 3D large FOV imaging is essentially need. In this paper, we developed a 3D PAT system using a hand-held linear ultrasound transducer array combined with a positioning system. The acquired 2D images and location information can be acquired in the real-time and then processed to realize the online 3D imaging using a dedicated 3D reconstruction algorithm. Both the simulation and in-vivo study have been conducted to verify the feasibility of the proposed system. The result of 3D PAT imaging on human wrist shows clear outline of the blood vessel with complete structure in 45 mmx38 mmx20 mm volume size, which well proved its feasibility for clinical applications in the future.

The Voxel-Based Nearest Neighbor method (VNN) is the most commonly used three-dimensional ultrasound (US) image reconstruction algorithm for spine imaging on scoliotic patients. The neural radiance field (NeRF) method is a newly developed reconstruction algorithm in computer vision area, which utilizes a weighted multilayer perceptron to rebuild the volumetric density and color of a 3D scene. The objectives of this study are to apply the NeRF algorithm for the reconstruction of 3D US spine data and to evaluate the spinal curvature measurement from the reconstructed results. The conventional VNN and NeRF method were used to reconstruct 3D ultrasound volume data for 25 scoliotic subjects. The mean absolute difference (MAD) between different reconstruction methods and the correlation between US and radiographic measurements were calculated. The result demonstrated that the new 3D reconstruction algorithm based on the NeRF algorithm could provide higher-quality 3D US spine images with highlighted bone structures and lower noise, moreover lead to better estimation of spine curves with smaller errors comparing to the VNN method.

The 3D ultrasound reconstruction technology has led to a rapid development of ultrasound spine imaging in recent decades. However, the current imaging apparatus is bulky and not portable. The objective of this study is to develop a new compact and wireless system to offer the real-time visualized spine images during data acquisition. A portable and WI-FI based ultrasound scanner and a compact EM tracking system were assembled to acquire ultrasound transverse frames with location information which could be reconstructed into 3D spine image volume in real-time. The validation was implemented on the 2D coronal images of vertebra phantoms, and the in vivo data acquisition and reconstruction were demonstrated on volunteers. The result showed that the new system could provide reconstructed spine images in real time and the average errors of the reconstructed images were about 1mm (approximate to image pixel size).

The Voxel-Based Nearest Neighbor method (VNN) is one of the most commonly used reconstruction algorithms for a freehand ultrasound (US) acquisition system, however, it is time-consuming due to large scale of vector arithmetic. The objectives of this study are to develop 3D US reconstruction algorithms which can reduce the reconstruction time to the required range of real-time demonstration and processing and to improve the US image quality for higher brightness and contrast. The execution time, numerical difference comparison, brightness and contrast analysis were used to demonstrate the new methods. The result showed that the new FDP US imaging method could provide 3D spine images with faster reconstruction procedure and better image quality.