AO
AceOffer
·
Back
Waymo Common Problems

Ego Behavior Prediction Model System Design

System DesigneasyLast reported September 2025
By AceOffer · Updated September 2025 · Reported 1× across 190+ candidate reports

Understanding the Problem

Design a machine learning model system for predicting the ego vehicle's future behavior in an autonomous driving context. The design should cover: (1) what inputs the model takes (e.g., sensor data, map information, surrounding agent states), (2) what outputs the model produces (e.g., predicted trajectories, maneuver intents), (3) how to quantify and represent uncertainty in the predictions, and (4) what outputs/contracts are appropriate to expose to downstream consumers (e.g., planning, motion control modules). The question is highly role-specific for a Behavior Prediction engineer and is probed in detail rather than breadth.

Functional Requirements

Structured requirements coming soon. For now, see the full problem statement above and the deep-dive prompts below.

Non-Functional Requirements

Latency, throughput, availability, consistency targets — being authored.

The Set Up

Defining the Core Entities

Core entities (Request, Batch, Worker, Cache, etc.) — being authored.

The API

POST /endpoint → describe request shape GET /endpoint → describe response shape (API spec being authored)

High-Level Design

Component diagram + walkthrough mapping each functional requirement to a system flow — being authored.

Potential Deep Dives

These are the directions the interviewer is likely to push you. Each one has multiple valid solutions at different quality tiers.

1)What specific outputs would you expose to downstream consumers, and what does the data contract look like? (when: Candidate describes model outputs)

Bad
Naive approach with serious trade-off — being authored.
Good
Solid baseline with reasonable trade-offs — being authored.
Great
Production-grade approach with explicit trade-off rationale — being authored.

2)How would you quantify uncertainty in the predictions? How do you distinguish different types of uncertainty? (when: Candidate mentions uncertainty)

Bad
Naive approach with serious trade-off — being authored.
Good
Solid baseline with reasonable trade-offs — being authored.
Great
Production-grade approach with explicit trade-off rationale — being authored.

3)What inputs does the model take, and why? (when: Candidate describes inputs)

Bad
Naive approach with serious trade-off — being authored.
Good
Solid baseline with reasonable trade-offs — being authored.
Great
Production-grade approach with explicit trade-off rationale — being authored.

What is Expected at Each Level?

L4 / Mid-level
Cover happy path. Clarify scope. Identify the obvious bottleneck. Pick a reasonable storage and reasonable scaling approach.
L5 / SeniorTarget
All of the above plus: explicit failure handling, durability vs latency trade-offs, choose the right batching/caching strategy, articulate why.
L6 / Staff+
All of the above plus: organizational concerns (rollout, migration, on-call), quantitative analysis, multi-region considerations, what could go wrong with the proposed solution at 10x scale.

Insider Notes

**Common mistakes:** Treating this as a generic ML system design without grounding in autonomous driving specifics (ego vs. agent distinction, map context); Outputting only a single deterministic trajectory instead of a multimodal distribution; Not addressing uncertainty quantification at all, or conflating aleatoric and epistemic uncertainty; Ignoring the downstream data contract — failing to specify what the consuming module (planner) actually needs **What passers do:** Candidate with strong domain knowledge (AV-related PhD/internship) who could address inputs, outputs, uncertainty, and downstream contracts fluently; Structured the answer around a production ML pipeline with clear module boundaries; Proactively discussed multimodal trajectory outputs and calibration of uncertainty estimates **Edge cases probed:** How to handle highly ambiguous scenarios (e.g., unprotected left turns, merges) in the output distribution; How downstream consumers should handle low-confidence or high-uncertainty predictions; What happens when ego is in an out-of-distribution environment (e.g., construction zones) **Alternative approaches:** Single deterministic trajectory output (Simpler interface for downstream consumers but loses multimodal uncertainty; fails in ambiguous scenarios like unprotected turns or merges; not suitable for safety-critical planning.); Occupancy grid / probabilistic map output (More general representation than explicit trajectories; easier to aggregate multiple agents; but loses interpretability of discrete intent modes and harder to use directly in trajectory-based planners.); Conditional imitation learning / policy-based prediction (Can capture complex driving styles; but training distribution mismatch (covariate shift) and harder to quantify uncertainty explicitly.)
Waymo · System Design · Last reported September 2025