Large Model Deployment System Design

Problem Overview

You have a large ML model (50GB–500GB in size) that needs to be deployed as quickly as possible to 1000 internal machines on a network with limited bandwidth. Design a system to accomplish this efficiently. Your solution must include: (1) an architecture/topology for distributing the model, (2) a file-chunking strategy,…

• 2 candidate-reported follow-ups — the exact probes interviewers asked, with the trigger for each
• The rest of the problem statement — full requirements, constraints, and edge cases
• Approach and trade-offs — what passing candidates did, and the mistakes that sink people