Guangxu Zhu

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments

Fine-tuning LLMs is necessary for various dedicated downstream tasks, but classic backpropagation-based fine-tuning methods require substantial GPU memory. To this end, a recent work, MeZO, which relies solely on forward passes to fine-tune LLMs, significantly reduces GPU requirements at the cost of…

Bridging Large-Model Reasoning and Real-Time Control Via Agentic Fast-Slow Planning

Think in Cloud, Look at Edges: Semantic-Driven Query Decomposition for Efficient Video Reasoning

Long video understanding faces a critical dilemma: cloud-based Large Multimodal Models (LMMs) offer superior reasoning but suffer from prohibitive bandwidth costs and latency, while edge-based solutions sacrifice perception accuracy for speed. Current collaborative approaches attempt to bridge this …

Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectory

To improve the generalization of the autonomous driving (AD) perception model, vehicles need to update the model over time based on the continuously collected data. As time progresses, the amount of data fitted by the AD model expands, which helps to improve the AD model generalization substantially

FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving

Street Scene Semantic Understanding (denoted as S3U) is a crucial but complex task for autonomous driving (AD) vehicles. Their inference models typically face poor generalization due to domain-shift. Federated Learning (FL) has emerged as a promising paradigm for enhancing the generalization of AD m

Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator

Learning-based street scene semantic understanding in autonomous driving (AD) has advanced significantly recently, but the performance of the AD model is heavily dependent on the quantity and quality of the annotated training data. However, traditional manual labeling involves high cost to annotate

Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization

Navigating autonomous vehicles in open scenarios is a challenge due to the difficulties in handling unseen objects. Existing solutions either rely on small models that struggle with generalization or large models that are resource-intensive. While collaboration between the two offers a promising sol

Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning

To support real-world decision-making, it is crucial for models to be well-calibrated, i.e., to assign reliable confidence estimates to their predictions. Uncertainty quantification is particularly important in personalized federated learning (PFL), as participating clients typically have small loca…

FedRC: A Rapid-Converged Hierarchical Federated Learning Framework in Street Scene Semantic Understanding

Street Scene Semantic Understanding (denoted as TriSU) is a crucial but complex task for world-wide distributed autonomous driving (AD) vehicles (e.g., Tesla). Its inference model faces poor generalization issue due to inter-city domain-shift. Hierarchical Federated Learning (HFL) offers a potential…

Integrating Sensing, Communication, and Computation in the Sky

Unmanned Aerial Vehicle (UAV)-mounted edge devices are particularly advantageous for federated edge learning (FEEL) due to their flexibility and mobility in efficient data collection. In UAV-assisted FEEL, sensing, computation, and communication are coupled and compete for limited onboard resources,…

Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving

While federated learning (FL) improves the generalization of end-to-end autonomous driving by model aggregation, the conventional single-hop FL (SFL) suffers from slow convergence rate due to long-range communications among vehicles and cloud server. Hierarchical federated learning (HFL) overcomes s…

Spectrum Allocation in Wireless Networks for Crowd Labelling

The massive sensing data generated by Internet-of-Things will provide fuel for ubiquitous artificial intelligence (AI), while tremendous labels are required for AI model training via supervised learning. To tackle this challenge, a novel framework of wireless crowd labelling is proposed that downloa…