Given multimodal sensor data (LiDAR, radar, etc.) and camera data from an autonomous vehicle, design one or multiple ML agents/systems to: (1) predict surrounding environment changes over time, and (2) anticipate and warn of impending dangers. The design should cover model selection, data processing pipeline, model architecture details, loss function design, training strategy/stages, and analysis of emergency/edge-case scenarios an autonomous vehicle might encounter.
Interviewer hints: Interviewers asked about model selection, data processing pipeline, model architecture details, loss function design, training stage design, and emergency/edge-case brainstorming — candidates should expect to cover all of these areas, not just high-level architecture.; The question was asked in two separate rounds with very similar content, suggesting the interviewer has a consistent set of topics they probe across data, loss, and training stages.
Alternative approaches: End-to-end imitation learning (Simpler pipeline by learning directly from expert demonstrations; however, harder to interpret, less generalizable to rare emergency scenarios, and requires large amounts of expert-labeled data.); Rule-based / model-based danger prediction (Highly interpretable and reliable for known scenarios, but does not generalize well to novel or complex environments and requires extensive manual rule engineering.); Reinforcement learning agent (Can learn optimal policies in simulation; difficult to train safely in real environments, requires careful reward shaping, and may struggle with rare real-world edge cases.)