Human Annotation with High-Dimensional Input

Problem Overview

You are given a labeled dataset produced by human annotators of varying quality. The dataset contains binary labels and annotators fall into (at least) three quality tiers (bad, mid, high). A black-box model and training code are provided; running them produces a baseline performance number. Your tasks are: (1) train…

• 4 candidate-reported follow-ups — the exact probes interviewers asked, with the trigger for each
• The rest of the problem statement — full requirements, constraints, and edge cases
• Approach and trade-offs — what passing candidates did, and the mistakes that sink people