About Me
I am a Ph.D. candidate in Electrical Computer Engineering at The University of Texas at Austin advised by Prof. Atlas Wang at VITA group.
I work closely with Prof. Marco Pavone and Prof. Yue Wang on 3D end-to-end models with robust generalization capabilities; with Prof. Achuta Kadambi on recovering 3D/4D signals that capture the space-time structure of our world from casually captured data; and with Prof. Callie Hao on hardware-software co-design.
I was the awardees of Qualcomm Innovation Fellowship 2022.
Recent News
- Our NeurIPS'24 (LightGaussian) is selected as spotlight presentation.
- Our Symbolic Visual RL was accepted by IEEE Trans. PAMI.
- Our IROS'24 (Multi-modal 3DGS SLAM) is selected as oral pitch finalist presentation.
- Our CVPR'24 (Feature-3DGS) is selected as highlight presentation.
- Our CVPR'23 (NeuralLift-360) is selected as highlight presentation.
- I was one of the awardees of the Qualcomm Innovation Fellowship (North America) 2022 (QIF 2022). Innovation title: "Real-time Visual Processing for Autonomous Driving via Video Transformer with Data-Model-Accelerator Tri-design".
- We won 3rd place in the University Demo Best Demonstration at the 59th Design Automation Conference (DAC 2022). We demo for a multi-task vision transformer on FPGA.
- Our CVPR'22 (CADTransformer) is selected as oral presentation.
- Our paper for CVPR'20 (Cascade Cost Volume) is selected as oral presentation.
Commitment
- I am dedicated to collecting high-quality, accessible online courses in artificial intelligence, machine learning and computer vision. Check the note page: LINK.
- I dedicate 1 hour weekly (2:00pm-2:30pm, 2:30pm-3:00pm, PST on Friday) to mentor and guide students from underrepresented groups or those in need. Interested? Fill out this form.
Researches and Projects
Humans possess the remarkable ability to perceive and interact with their environment. This ability
is driven by an internal understanding of how the scene's structure is formulated and the inherent
properties of the environment. To equip future intelligent machines with this capability, it is essential
to perceive geometry directly from visual inputs for interaction with the physical world, rather than
relying on offline algorithms for preprocessing camera poses, which limits the scalability of foundation
models for safe and reliable planning.
The next-generation learning algorithms that equipped with visual sensors, shall inherently perceive geometric structures that
pre-train 3D foundation models that leverage Internet-scale video data; align with existing VLMs for reliable reasoning and planning
by leveraging physical geometry from visual inputs; investigate novel architectures that can efficiently
process, interpret, and reason over high-resolution visual streams in the temporal dimension.
- Large Spatial Model (NeurIPS'2024).
- Feature-3DGS (CVPR'2024, Highlight).
- NeRF-SOS (ICLR'2023).
The emergence of reasoning or strong generalization capabilities within foundation models are built upon the ability of processing and compressing large-scale data into well-designed, scalable models. However, 3D learning typically requires lengthy, modular, and non-differentiable pipelines for calibrating image or video data. This paradigm significantly hinders the scaling of 3D models to learn from web-scale, unannotated video data that lacks image annotations and camera poses. My researches address a practical and compelling challenge: 3D reconstruction from pose-free, unannotated image data.
- InstantSplat (CVPRW'2024).
- LightGaussian (NeurIPS'2024, Spotlight).
My research tackles the challenge of data sparsity in photorealistic 3D digital environments, where dense scene capture with annotated poses is often unavailable. By combining Geometric Principles with Generative Priors—learned from large datasets—My reseaches fill in missing information using statistical patterns in shape and appearance. This approach leverages both the deterministic nature of geometry and the probabilistic power of generative models, enabling architectures to effectively learn from limited data.
- Cascade Cost Volume (CVPR'2020, Oral).
- Cas6D (3DV'2024).
- Few-Shot Gaussian Splatting (ECCV'2024).
- SinNeRF (ECCV'2022).
- NeuralLift-360 (CVPR'2023, Hightlight).
- DreamScene360 (ECCV'2024).
My research has been demonstrated on platforms such as Quest 3, implemented within IARPA projects, and integrated into multiple commercial products.
Selected Publications
Full publication list at Google
Scholar
Invited Talks
-
Empowering Machines to Understand 3D @ Stanford, ASU, JHU. October 2024.3D Computer Vision @ TAMU Guest Lecture. October 2024.From Efficient 3D Learning to 3D Foundation Models @ UCLA and CalTech. October 2024.Towards Universal, Real-Time 3D Construction and Interaction @ TAMU AI Lunch. October 2024.Spatial Intelligence via Reconstruction, Distillation, and Generation @ Shanghai AI Lab. July 2024."Streamlined 3D/4D: From Hours to Seconds to Millisecond @ Google Research, VALSE Webinar . May 2024."Streamlined 3D/4D: From Hours to Seconds to Millisecond @ Google Research. May 2024."Real-Time Few-shot View Synthesis w/ Gaussian Splatting @ IARPA WRIVA Workshop. April 2024."Data-efficient and Rendering-efficient Neural Rendering @ IFML Workshop on Gen AI. November 2023."Unified Implicit Neural Stylization @ Xiamen University; Kungfu.ai.. July 2022.
Experience
-
Meta, Reality Lab, Burlingame:Research Intern (year of 2024).
-
NVIDIA Research (remote):Research Intern (year of 2024).
-
Meta, Reality Lab, Redmond:Research Intern (year of 2023).
-
Google AR, San Francisco:Research Intern (year of 2022).
-
Alibaba Group, Hangzhou:Senior Algorithm Engineer(2019 - 2021).
Services
-
Journal Reviewers:TPAMI, TIP, IJCV, Neurocomputing.
-
Conference Reviewers:NeurIPS 22/23, ICML 22/23, CVPR 22/23, ICCV 21/23, AAAI 21, ICME 2019.