Jiaqi Peng

We present a sim-to-real navigation diffusion policy that can achieve cross-embodiment generalization in dynamic, cluttered and diverse real-world scenarios.

Although existing 3D perception algorithms have demonstrated significant improvements in performance, their deployment on edge devices continues to encounter critical challenges due to substantial runtime latency. We propose a new benchmark tailored for online evaluation by considering runtime latency. Based on the benchmark, we build a Latency-Aware 3D Streaming Perception (LASP) framework that addresses the latency issue through two primary components: 1) latency-aware history integration, which extends query propagation into a continuous process, ensuring the integration of historical feature regardless of varying latency; 2) latency-aware predictive detection, a module that compensates the detection results with the predicted trajectory and the posterior accessed latency. By incorporating the latency-aware mechanism, our method shows generalization across various latency levels, achieving an online performance that closely aligns with 80% of its offline evaluation on the Jetson AGX Orin without any acceleration techniques.

Objective Video-based multiple object tracking is one of the essential tasks in computer vision like automatic driving and intelligent video surveillance system.Most of the multiple object tracking methods tend to obtain object detection results first.The integrated strategies are used to link detection bounding boxes and form object trajectories.Current object detection contexts have been developing recently.But,the challenging inconsistency issues are required to be resolved in multiple object tracking,which affected the multi-objects tracking accuracy.The multi-objects tracking inconsistency can be classified into three types as mentioned below:1) the inconsistency between the centers of the object bounding boxes and those object identity features.Many multiple object tracking methods are extracted the object re-identification (ReID) features at the object bounding boxes centers and these features are used to in associate with objects.However,those oriented ReID features are incapable to reflect the appearance of objects accurately due to the occlusion.The offsets are appeared between the best ReID feature extraction positions and bounding box centers.Current feature extraction strategy will lead to the spatial consistency problem.2) The inconsistency of the object center response between consecutive frames.Some objects can be detected and tracked in the contexted frames due to the occlusion in videos.It causes consecutive frames loss and the inconsistency between the object-center-responsed heatmaps of two consecutive frames.3) The inconsistency of the similarity assessment in the training process and testing process.Most of association step is considered as a classification problem and the cross entropy loss is used to train the model while the inter-object relations are ignored in the testing process.The feature cosine similarities of each pair of objects are used to associate them.To improve the accuracy of tracking,we facilitate a multiple object tracking method based on consistency optimization.