63 pubs · 675 cites · h-index 15
Three-dimensional photoacoustic imaging (3D PAI) has shown great potential in preclinical studies and clinical applications, but sparse sensor arrays commonly limit both spatial sampling and the field of view (FOV). Freehand scanning could overcome these limitations through anatomy-adaptive scanning that extends spatial coverage and provides complementary views beyond the fixed array aperture, but its flexibility is conventionally constrained by the need for motor feedback or external pose tracking. Here we introduce PA-SfM, a tracker-free structure-from-motion (SfM) framework that recovers multiple poses directly from PA measurements, enabling freehand 3D PAI without external pose measurements. PA-SfM integrates a differentiable acoustic radiation model with hierarchical optimization and rigid array constraints to jointly estimate transformations between views and reconstruct 3D volumes. The performance of PA-SfM was demonstrated in numerical simulations, animal studies and human imaging experiments. Compared with motor encoder-based fusion of the mechanically scanned system, PA-SfM reduced vessel misalignment in expanded FOV reconstructions. Controlled quantitative validations yielded peak signal-to-noise ratio (PSNR) values over 40 dB and structural similarity index (SSIM) values about 0.98 relative to ground truth or known pose reference reconstructions. Freehand experiments further demonstrated robust pose recovery at high scan speeds, with inter-frame displacements of 9 mm at intervals of 0.1 s. These results support PA-SfM as a tracker-free approach to large-scale freehand 3D PAI and provide a basis for further evaluation in clinical settings.
Three-dimensional hemispherical photoacoustic computed tomography provides vital structural information for non-invasive preclinical research. However, speed-of-sound heterogeneity between the coupling water and biological tissue induces wavefront aberrations, resulting in tangential blurring and ghosting artifacts. Traditional compensation typically relies on simplistic ellipsoidal fitting, which fails to capture irregular tissue boundaries, or utilizes complex multimodal ultrasound hardware that significantly increases system cost.To address these limitations, we introduce an automated, low-cost signal-domain sensing framework for a 1,024-element array that achieves high-fidelity three-dimensional boundary extraction without manual intervention or auxiliary hardware. By establishing a robust Median/Median Absolute Deviation based statistical noise baseline and a channel-specific, self-adaptive strong signal threshold, the framework maps the irregular acoustic interface directly from raw data, effectively isolating biological signals from intense scattering spikes and high-amplitude optical diffuser-induced artifact arcs. This signal-centric approach establishes an equivalent dual-speed-of-sound model through temporal compression, realigning distorted wavefronts with minimal computational overhead.Validated through three-dimensional simulations, irregular phantoms, and in vivo mouse imaging, the algorithm significantly restored the morphological fidelity of hepatic microvasculature. Furthermore, stress tests confirmed exceptional resilience even when the signal-to-noise ratio was artificially degraded to 30.9 dB. This robust architecture enables high-fidelity three-dimensional imaging in acoustically challenging and high-interference biological environments.
Reveals improved energy harvesting in sustainable piezoelectric materials, suggesting eco-friendly advancements.
Demonstrates enhanced motility of switchable nanomotors under light, suggesting versatile applications.
Hierarchical exploration method for effective data collection and environmental monitoring in unknown settings, suggesting improved efficiency and flight stability.
The exponential growth of data traffic in modern data centers has driven the demand for ultra-high-bandwidth, low-power optical interconnects. While 200 Gbps per wavelength has become standardized, next-generation systems aim to exceed this benchmark without relying on power-hungry digital signal processors (DSPs). Leveraging the CMOS compatibility and scalability of germanium photodetectors (Ge-PDs) over traditional III-V-based counterparts, we report, for the first time, to the best of our knowledge, a 240-Gbps O-band optical interconnect over 2 km of standard fiber without any DSP and amplification in both optical and electrical domains. This demonstration employs a >70 GHz thin-film lithium niobate (TFLN) modulator and a 91 GHz Ge PD, establishing a viable path toward compact, energy-efficient, and high-speed heterogeneous integration for future optical transceivers.
This paper introduces a novel globally consistent dense RGB-D SLAM system designed to address the challenges of online dense reconstruction under fast camera motion. To achieve robust pose tracking and globally consistent map building, we present a random optimization pose estimation framework that integrates both geometric and photometric information. Furthermore, we propose a voxel revisitation-based loop closure detection scheme that relies solely on depth maps. Innovatively, we introduce a local curvature feature, to address loop closure detection when photometric information is unavailable, efficiently extracting 3D geometric information directly from 2D images. In addition, we leverage voxel revisitation observations to accelerate the search for potential loop closures. This is achieved by maintaining a dynamic active map and detecting voxel outflow or revisits, thereby enhancing loop closure detection efficiency. This method is the first voxel revisitation-based loop closure detection scheme that relies entirely on depth information. Combined with a random optimization pose estimation framework utilizing both geometric and photometric features, it adapts well to fast camera motion, enabling accurate and globally consistent dense map construction. Extensive evaluations on public datasets demonstrate the effectiveness of our method under fast camera motion and highlight its significant advantages over existing methods.
Multispectral photoacoustic tomography (PAT) can create high spatial and temporal resolution images of oxygen saturation (sO2) distribution in deep tissue. However, unknown distributions of photon absorption and scattering introduces complex modulations to the photoacoustic (PA) spectra, dramatically reducing the accuracy of sO2 quantification. In this study, a rigorous light transport model was employed to unveil that the PA spectra corresponding to distinct sO2 values can be constrained within separate convex cones (CCs). Based on the CC model, sO2 estimation is achieved by identifying the CC nearest to the measured data through a modified Gilbert-Johnson-Keerthi (GJK) algorithm. The CC method combines a rigorous physical model with data-driven approach, and shows outstanding robustness in numerical, phantom, and in vivo imaging experiments validated against ground truth measurements. The average sO2 estimation error is approximately only 3% in in vivo human experiments, underscoring its potential for clinical application. All of our computer codes and data are publicly available on GitHub.
Accurate and regularly updated global road maps have many applications in intelligent navigation and urban planning. Deep learning algorithms have recently shown promising road extraction results using various Earth observation data. However, limitations in sample annotation and domain discrepancies between different regions and sensors pose challenges to the applicability of existing road extraction methods for large-scale tasks. In this paper, we propose a novel GlobalRoadMapper scheme to extract global-scale roads at the submeter level from optical remote sensing images that only require limited labeled samples. The proposed GlobalRoadMapper is designed as a two-stage method integrating Supervised Domain Incremental and Unsupervised Domain Adaptation. The supervised domain incremental stage learns road features from the source domain. To deal with catastrophic forgetting when deep models learn new knowledge from new domains, a strategy that couples pre-domain replay and mean-teacher is proposed. The unsupervised domain adaptation stage expands knowledge learned from the source domain to unseen scenes. Multi-level domain adaptation is proposed at the image, feature, and prediction levels in the second stage to address the issue of weak cross-domain generalization. The effectiveness of GlobalRoadMapper was tested at 43 sites worldwide. Qualitative and quantitative results demonstrate that GlobalRoadMapper outperforms the existing methods for global-scale road extraction tasks. Furthermore, city-scale sub-meter road mapping was conducted in six cities from different regions worldwide. GlobalRoadMapper achieved visual results comparable to the manually annotated OpenStreetMap road networks. Overall, GlobalRoadMapper holds great potential for large-scale road extraction and can be adapted to create easily updatable road maps globally.
The convolutional neural network (CNN) method is widely used in the field of stereo matching, and common CNNs regress disparity information by forming cost volumes. However, during the construction of cost volume, various types of information are integrated and combined; these pieces of information are input into an aggregation network to enable full learning when there is no significant feature guidance, which is a hypodynamic approach. To address this problem, we propose a stereo-matching slice method. Our core idea is to improve the salience of cost features through slicing, enhance feature self-guidance and self-interaction, and alleviate the mismatch between network aggregation capability and cost information. First, we use a group-wise correlation stereo network as the baseline; integrate the slice idea into feature extraction, cost volume construction, and feature aggregation; and propose the Gwc-Slice. Specifically, the residual extraction network is divided into seven groups of feature extractors by channel, the outputs of the feature extractors are used to construct slice cost volumes using the adaptive slice method, and different slice cost volumes are aggregated through different scales to enhance feature interactions and guidance between slice cost volumes. Finally, a slice network is designed to further optimize Gwc-Slice and verify the effectiveness of the slice method. The test results based on the SceneFlow, KITTI2015, KITTI2012, and Middlebury datasets show that our approach can significantly improve the prediction accuracy of the baseline and achieve competitive results.
• The first unsupervised LiDAR-depth fusion depth completion method, named LidarDepthNet. • Only depth input is required, making it suitable for application in darkness. • LidarDepthNet effectively fuses heterogeneous depth to obtain high-quality depth images. • LidarDepthNet outperformed comparative methods under real-world and synthetic datasets. • The first dataset for heterogeneous depth information fusion was provided. In this work, a depth-only completion method designed to enhance perception in light-deprived environments. We achieve this through LidarDepthNet, a novel end-to-end unsupervised learning framework that fuses heterogeneous depth information captured by two distinct depth sensors: LiDAR and RGB-D cameras. This represents the first unsupervised LiDAR-depth fusion framework for depth completion, demonstrating scalability to diverse real-world subterranean and enclosed environments. To facilitate unsupervised learning, we leverage relative rigid motion transfer (RRMT) to synthesize co-visible depth maps from temporally adjacent frames. This allows us to construct a temporal depth consistency loss, constraining the fused depth to adhere to realistic metric scale. Furthermore, we introduce measurement confidence into the heterogeneous depth fusion model, further refining the fused depth and promoting synergistic complementation between the two depth modalities. Extensive evaluation on both real-world and synthetic datasets, notably a newly proposed LiDAR-depth fusion dataset, LidarDepthSet, demonstrates the significant advantages of our method compared to existing state-of-the-art approaches.
TIMD4hiMHCⅡhi macrophages and their signature gene Retnla (RELMα) play a protective role in delaying heart failure progression.
Thin‐film lithium niobate (TFLN)‐based electro‐optic modulators have extensive applications in broadband optical communications due to their broad bandwidth, high extinction ratio, and low optical loss. However, compared with their silicon and indium‐phosphide (InP)‐based counterparts, TFLN exhibits lower modulation efficiency. Simultaneously achieving low driving voltage and a wide modulating bandwidth poses a significant challenge. To address this limitation, this paper proposes a transparent conductive oxide film into the device, resulting in an ultra‐high modulation efficiency of 1.02 V cm. The fabricated composite electrode not only attains high modulation efficiency but also sustains a high electro‐optic bandwidth, as evidenced by the 3 dB roll‐off at 108 GHz and the transmission of PAM‐4 signals at 224 Gbit s −1 . The fabricated device offers novel solutions for low‐cost, high‐performance modulators, thereby facilitating the downsizing of TFLN‐based multichannel optical transmitter chips.
Adiabatic mode evolution waveguides (AMEWs) are widely utilized in integrated photonics, including tapered waveguides, edge couplers, mode converters, splitters, etc. An analytical theory and a novel AMEW design algorithm are developed to create shortcuts to adiabaticity (STA). With the new algorithm, we demonstrate a broadband and highly efficient polarization splitter-rotator (PSR) on a lithium-niobate-on-insulator (LNOI) platform with an LN thickness of 500 nm. The fabricated PSR, with a total length of 2 mm, exhibits an insertion loss (IL) of 0.8 dB and a polarization extinction ratio (ER) of 12 dB over a wavelength range exceeding 76 nm.
Cross-modal analysis of the same whole brain is an ideal strategy to uncover brain function and dysfunction. However, it remains challenging due to the slow speed and destructiveness of traditional whole-brain optical imaging techniques. Here we develop a new platform, termed Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN), for non-destructive, high-speed, 3D imaging of ex vivo rodent, ferret, and non-human primate brains. Using an optimally designed image acquisition scheme and an accompanying machine-learning algorithm, PATTERN extracts signals of genetically-encoded probes from photobleaching-based temporal modulation and enables reliable visualization of neural projection in the whole central nervous system with 3D isotropic resolution. Without structural and biological perturbation to the sample, PATTERN can be combined with other whole-brain imaging modalities to acquire the whole-brain image with both high resolution and morphological fidelity. Furthermore, cross-modal transcriptome analysis of an individual brain is achieved by PATTERN imaging. Together, PATTERN provides a compatible and versatile strategy for brain-wide cross-modal analysis at the individual level.
In recent years, high-performance thin-film lithium niobate (TFLN) electro-optic (EO) modulators boost the fast development of highly integrated, low loss, and large comb spacing EO frequency combs. Furthermore, ultra-short optical pulse trains (USOPTs) can be generated by the temporal domain compression of the optical frequency comb, which play an essential role in photonic sampling analog-to-digital conversion. Here, we demonstrate a flat and broadband EO frequency comb based on a packaged TFLN chip including a monolithic integrated intensity modulator, a phase modulator, and edge couplers. The 25 comb lines with a power fluctuation less than 3 dB are presented successfully. Moreover, we obtain a 10 GHz repetition rate USOPT, the pulse width of which is compressed to 2.67 ps. Our device may find its applications in the fields of ultrafast measurement, wavelength-division-multiplexing optical communication, or high-precision photonic sampling.
Broadband, low-drive voltage electro-optic modulators are crucial optoelectronic components in the new-generation microwave photonic links and broadband optical interconnect network applications.In this paper, we fabricate a low-loss thinfilm lithium niobate complementary dual-output electro-optic modulator chip with a 3 dB electro-optic bandwidth of 59 GHz and a half-wave voltage (Vπ) of 2.5 V.The insert-loss of the packaged modulator is 4.2 dB after coupling with polarizationmaintaining fiber.The complementary dual-output modulator also shows a common-mode rejection ratio of 18 dB and a signal enhancement of 6.2 dB when adapted in microwave photonic links, comparable to commercial bulk lithium niobate devices.
BACKGROUND AND OBJECTIVE: Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms. METHODS: We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs. RESULTS: is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data. CONCLUSIONS: This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.
Unmanned aerial vehicles (UAVs) are extensively employed for urban image captures and the reconstruction of large-scale 3D models due to their affordability and versatility. However, most commercial flight software lack support for the adaptive capture of multi-view images. Furthermore, the limited performance and battery capacity of a single UAV hinder efficient image capturing of large-scale scenes. To address these challenges, this paper presents a novel method for multi-UAV continuous trajectory planning aimed at the image captures and reconstructions of a scene. Our primary contribution lies in the development of a path planning framework rooted in task and search principles. Within this framework, we initially ascertain optimal task locations for capturing images by assessing scene reconstructability, thereby enhancing the overall quality of reconstructions. Furthermore, we curtail energy costs of trajectories by allocating task sequences, characterized by minimal corners and lengths, among multiple UAVs. Ultimately, we integrate considerations of energy costs, safety, and reconstructability into a unified optimization process, facilitating the search for optimal paths for multiple UAVs. Empirical evaluations demonstrate the efficacy of our approach in facilitating collaborative full-scene image captures by multiple UAVs, achieving low energy costs while attaining high-quality 3D reconstructions.
Real-time 3D reconstruction combined with MAVs has garnered significant attention in a variety of fields, including building maintenance, geological exploration, emergency rescue, and cultural heritage protection. While MAVs possess the advantages of speed and lightness, they also exhibit strong image blur and limited computational resources. To address these limitations, this paper presents a novel approach for onboard, depth-only, real-time 3D reconstruction capable of accommodating fast-moving MAVs. Our primary contribution is a dense SLAM system that combines surface hierarchical sparse representation and particle swarm pose optimization. Our system enables the robust tracking of high-speed camera motion and facilitates scaling to large scenes without being constrained by GPU memory resources. Our robust camera tracking framework is capable of accommodating fast camera motions and varying environments solely by relying on depth images. Furthermore, by integrating path planning methods, we explore the capabilities of MAV autonomous mapping in unknown environments with restricted lighting. Our efficient reconstruction system is capable of generating highly dense point clouds with resolutions ranging from 2 mm to 8 mm on surfaces of different complexities at rates approaching 30 Hz, fully onboard a MAV. We evaluate the performance of our method on both datasets and real-world platforms and demonstrate its superior accuracy and efficiency compared to existing methods.