🤔 How might LLMs store facts? Large language models (#LLMs) like GPT-3 appear to have memorized many facts, which are likely stored in the multilayer perceptrons (MLPs) that make up a large portion of the model parameters. 🟢 The key steps inside an MLP block are: 1. Matrix multiply the input vector by a large weight matrix W_up, adding a bias B_up 2. Apply a ReLU non-linearity (clipping negative values to 0)  3. Matrix multiply by a smaller weight matrix W_down, adding a bias B_down 4. Add the result to the original input vector 🟢 If the input vector encodes "Michael Jordan", the MLP computations will: 1. Detect this via the AND-like behavior of the ReLU 2. Add the "basketball" direction to the output vector 3. The final result will encode "Michael Jordan" + "basketball" So the MLP can effectively store the fact that "Michael Jordan plays basketball" by associating the full name with the sport in its weight matrices. 👉 https://lnkd.in/dBeDBZrH ♻️𝒌𝒊𝒏𝒅𝒍𝒚 𝒓𝒆𝒑𝒐𝒔𝒕♻️

69

🎇 On National Day, I went for a leisurely drive in San Francisco and ended up "stress-testing" a Waymo self-driving car on the road. 🚗 As an autonomous driving practitioner, who wouldn't be curious about the real-time robustness of the cutting-edge Waymo One driving system? While cruising downtown, I accidentally noticed a Waymo car tailgating me. While this isn't unusual for SF citizens, a wild, "evil" idea suddenly hit me: Why not directly "adversarially attack" the world's autonomous driving status quo? I executed an unexpected maneuver—suddenly reversing—to see how it would react. 🌟 The response was stellar! At the moment I reversed, Waymo One honked instantly—quicker than any human could—activated its hazard lights, and backed away to maintain a safe distance. This reckless move on my part served as an edge case to test the algorithm's robustness under extreme conditions and, potentially, could be a challenging training sample to enhance Waymo's future autonomous systems. 💫 Thrilled to personally experience how current cutting-edge autonomous algorithms handle rare driving behaviors—and how stable and safe Level 4 autonomy is in dealing with diverse scenarios. However, it also prompted deep reflection as an AI researcher in this field: 🤔 In an industry with little room for error, how can we ultimately avoid or minimize issues that AI fails to handle? 💡 I believe two research directions are particularly promising for achieving Level 5 level autonomy in future mobility systems: - 1️⃣ Development and deployment of vehicle-to-everything (V2X) cooperative systems (including V2V, V2I, V2P, etc). Our initial studies (e.g., V2X-ViT, ECCV'2022 arxiv.org/abs/2203.10638) show that in scenarios with severe occlusions or noise, such cooperative systems can significantly enhance the robustness of perception systems, thereby eventually improving traffic safety. - 2️⃣ Adversarial scenarios generation (including Sim-to-Real, generative modeling). Research done by my colleagues at UCLA (V2XP-ASG, ICRA'2023 https://lnkd.in/gu5nKVHD) shows that adversarial learning techniques can effectively simulate adversarial scenarios, greatly improving model robustness in complex "corner case" situations. Of course, it's often infeasible to collect such collision scenarios. 👨‍🏫 As a new Assistant Professor of CS at Texas A&M University, I will lead a group focusing on these exciting research directions, which can be a proactive approach to reducing accidents and improving safety for all. 🔥 I look forward to future collaborations with governments, academics, and companies to research and develop data and algorithms that can help enhance the safety of vulnerable road users, especially seniors and children. We envision a people-centered intelligent transportation system in the future. Interested in these topics? Let's connect and discuss further! #AutonomousVehicles #AI #MachineLearning #SmartCities #Transporation #Humanity #Mobility

145

3 Comments

Check out this 4-part on-demand webinar series that go over best practices to overcome challenges with large scale modeling. * Simulink Components & Interfaces * MATLAB Projects: From Dependency Analysis to Batch Processing * Managing Model Parametric Data * Speeding up Simulink Simulations https://spr.ly/6049g3DKz

3

At the height of #autonomousvehicle hype, companies such as NVIDIA, Qualcomm and Mobileye were front-runners dictating next-generation vehicle architecture. As more full-fledged software-defined vehicles (#SDV) evolve, the competitive landscape is rapidly changing. Knowing #ADAS and #IVI might not be enough to win the #SDV race. Asif Anwar TechInsights Brian Carlson NXP Semiconductors Aish Dubey Renesas Electronics #S32N55, #RCAR5

99

2 Comments

NHTSA is investigating Waymo safety "after getting 22 reports of Waymo vehicles either crashing or doing something that may have violated traffic laws." This includes "17 crashes and five other reports of possible traffic law violations. No injuries were reported." After the spate of wrong-way driving videos and the rear-end mishap it seems appropriate for NHTSA to take a closer look and find out how these vehicles are performing according to the "undue risk" criterion. Readers should note that this is a different criterion than the "positive risk balance" net risk claim commonly made by Waymo, but in fact undue risk is the regulatory criterion for recalls. ODI Resume with investigation details here: https://lnkd.in/eWhTnAHU https://lnkd.in/exyW2BAY

67

3 Comments

I successfully finished the 'Generative AI with Large Language Models' course and earned certification from DeepLearning.AI. Key topics : - Transformer Architecture - Prompt Engineering & Instruction Fine Tuning - LLM Scaling Laws, Chinchilla paper - Parameter Efficient Fine-tuning (PEFT) - Low-Rank Adaptation of LLMs (LoRa) - Single Task and Multi-Task LLM Fine-tuning Advanced techniques such as: - RLHF, RLAIF by Constitutional AI - LLM Optimization - Distillation, Quantization, Pruning - LLM's role in powering Apps via Orchestration Library - Retrieval Augmented Generation (RAG) Framework for accessing external data others: Chain-of-thoughts prompting, ReAct, and Augmented LLM by Program-aided Language Model (PAL). Also, gained insights from one of the exciting areas of the course i.e. Proximal Policy Optimization (PPO) presented by Dr. EK.

28

6 Comments

AI NEWS SERIES Applied Intuition’s Secures $300m for AI in Vehicle Manufacturing Product: - Applied Intuition offers a comprehensive ADAS/AD development toolchain, integrating simulation and data products for rapid, safe automotive software development. - Key products include off-road autonomy stack, generative AI for simulation scenarios, and hardware-in-the-loop testing software. Company: - Founded in 2017, headquartered in Mountain View, CA, Applied Intuition has raised $901.5M from investors like Andreessen Horowitz and Porsche Investments. Impact: - The company's advanced simulation technology enables more efficient, safer development cycles in the automotive industry, shaping future transportation and influencing industry standards. sources: https://lnkd.in/e6VBnZYG https://lnkd.in/e5naFTXu #technology #innovation #future #artificialintelligence #business

1

Autonomous Vehicles: Advancing Fast, But Are We Safe? The race for self-driving cars is on, but are we prioritizing safety and security? Join my talk at the LA 2024 Autonomous Vehicles Conference as we delve into the industry's biggest hurdles. We've all seen the Hollywood hacking scenes, but what about the unforeseen safety risks lurking around the corner? Traditional safety standards won't cut it. We need a revolutionary approach to ensure passenger well-being in an autonomous future. This session is a must-watch for anyone invested in the future of transportation! Want to learn more? * Grab my book, "A Brief History of Autonomous Vehicles" for a deeper dive. * Have questions? Leave them in the comments below! ⬇️ #AutonomousVehicles #Safety #Security #TransportationFuture #automotivecybersecurity #ios21434 #iso26262

6

1 Comment

Wondered how to derive validation targets for AI components from acceptance criteria for ADAS/ADS according to SOTIF? I’ll be presenting a paper [1] (with Hiroshi Kuwajima [2]) at the SAE WCX'24 [3] on deriving hazardous error sequence patterns that AI components may produce from vehicle-level hazardous behaviors. The key idea is to use weakest precondition computation and conservative approximations. The approach helps focus testing on estimating parameters needed for the analytical model instead of trying to cover a range of behaviors in testing to assure against specific hazardous behaviors. BTW, the concept of error sequence patterns introduced in our work is now part of the upcoming ISO/PAS 8800 on Safety and AI. [1]  Paper: https://lnkd.in/gqQ7Mpvn Pre-print: https://lnkd.in/gZX6kWGk [2] https://lnkd.in/gUg3PK-Z [3] https://wcx.sae.org

21

Quick video on remote operations and autonomous vehicles: IMHO whether a robotaxi/robotruck is autonomous hinges critically on the responsibility for safety. If a remote operator can be blamed for causing a mishap, then the vehicle is not really autonomous. But a remote operator providing suggestions is a matter of system engineering tradeoffs and might be fine. Waymo & Cruise both position themselves as autonomous robotaxis and have repeatedly said that the remote operator provides suggestions while the vehicle is completely responsible for ensuring driving safety while in autonomous operation. Anyone who said, for example, that frequent "phone home" events for Cruise made their robotaxi somehow non-autonomous I think was missing the nuances of the situation. That certainly calls scalability into question, but not necessarily safety. (In fact, a significant contribution to the pedestrian dragging mishap was that the vehicle did NOT pause operations to phone home when it should have, demonstrating their autonomy had a serious safety issue.) However, let's consider the case of a Waymo incident in which the Waymo ran a red light associated with a moped rider losing control, falling, and sliding in January. Industry apologists might try to blame this on "human error" (which I disagree with) or a poor remote operations approach (which sounds more appropriate). But I would also say this mishap reveals an insincere industry narrative in which a human is blamed to draw criticism away from a robotaxi -- which previously was said to be in charge of safety to the degree a human error could not compromise. Either an autonomous robotaxi is responsible for running a red light or there is a Wizard of Oz setup with a person behind the curtain making life or death decisions. Which is it? Video outlining the topic of remote operations here: https://lnkd.in/eW3e7xeB

43

3 Comments

This is a great paper and one of the quickest ways to understand the Cruise robotaxi incident.

20

2 Comments

🌟 Humans, Robots and Autonomous Vehicles - at Auto UI 2024! 🌟 Exciting opportunity for my extended HRI and Autonomous Vehicle Interaction research community! Join me for the "Symbiotic - Workshop on Transferability between Human-Robot Interaction (HRI) and Autonomous Vehicle-Other Road Users Interactions Research" at the upcoming Auto UI 2024 conference, hosted at Stanford University. 🔍 Workshop Focus: Explore the intersections of interactions with Robots and Autonomous Vehicles (AVs), foster discussions and collaborations on the transferability of methodologies and insights between these fields. 🤖 Who Should Attend? Researchers and professionals in HRI and AV interaction domains—roboticists, AI specialists, UX designers, ethicists, policymakers, and forward-thinking industry innovators. 📝 Call for Contributions: We're looking for insights that could span across HRI and AV interaction realms: * Position papers and empirical studies on HRI and AV interaction. * Innovative models and frameworks  * Theoretical explorations and ethical paradigms.  Submissions can range from short work-in-progress to comprehensive research papers. 🎤 Features of the Workshop: Engaging panel discussions with experts. Opportunities to present your research. Collaborate on a multi-author paper summarizing the workshop's insights. 📅 Mark Your Calendars! Paper Submission Deadline: August 25 Organized by: Paul Schmitt (Torc Robotics) Swapna Joshi, PhD (Northeastern University) Bruce N. Walker (Georgia Tech) Azra Habibovic (Scania) For more details and to register, visit Automotive UI '24. Special thanks to MassRobotics for supporting this work!   Looking forward to seeing many of you there! #AutoUI2024 #HumanRobotInteraction #AutonomousVehicles #MassRobotics #InteractionResearch #ResearchCollaboration #Innovation #Technology https://lnkd.in/ee9EX8bK https://lnkd.in/ebcEQj3m

46

1 Comment

Did you ever wonder how an autonomous vehicle steers itself to self-navigate a route? 🎙️ Check out Episode 19 of the DRIVEN podcast, where I explain autonomous vehicle self-navigation. In this episode I explore how autonomous vehicles leverage position and heading information, path planning, and algorithms like PID control and pure pursuit to navigate roads with precision. Join me on this DRIVEN podcast journey for an evolving conversation on the future of transportation and how we're getting there with the algorithms that drive vehicles. #AutonomousVehicles #Technology #Innovation https://lnkd.in/e5dusvGa

9

👉 Apply fundamental Simulink techniques for real-life dynamic physical systems modeling. Dive into modeling systems with multiple components and deepen your understanding of how Simulink runs simulations behind the scenes. https://spr.ly/6042l9n6j

2