Job Description:
• Design, implement and own ML metrics and evaluation pipelines spanning offline model evaluation, simulation and on-road performance.
• Build and maintain test, regression and hillclimbing suites that gate model and stack releases, including automated triage of regressions to root cause.
• Drive model improvement through loss analysis, error mining, and data balancing/curation strategies for training and evaluation sets.
Requirements:
• Bachelor's or Master's degree in Robotics, Computer Science, Statistics, or a related field with strong mathematical and engineering foundations.
• A minimum of 2 years building evaluation, metrics, or data analysis systems for ML in production.
• Proficiency in Python (NumPy/Pandas or equivalent dataframe tooling) with experience in Linux environments.
• Familiarity with basic concepts in Machine Learning (losses, train/eval splits, common failure modes) and basic Perception and Planning concepts in Autonomous Driving.
• Proficiency in Go or C++ (desirable).
• Familiarity with experiment tracking and evaluation tooling such as MLflow, Weights & Biases, or in-house equivalents (desirable).
• Familiarity with statistical methods for A/B comparison, regression detection and noisy-metric analysis (desirable).
• Familiarity with data mining and curation at scale (embedding-based retrieval, active learning, auto-labeling) (desirable).
• Familiarity with visualization and dashboarding tools (Plotly, Grafana, Streamlit or similar) (desirable).
Benefits:
• Comprehensive healthcare suite including medical, dental, vision, life, and disability plans. Domestic partners who have been residing together at least one year are also eligible to participate.
• Health Savings and Flexible Spending Healthcare and Dependent Care Accounts available.
• Rich retirement benefits, including an immediately vested employer safe harbor match.
• Generous paid parental leave as well as a phased return to work.
• Flexible vacation policy in addition to paid company holidays.
• Total Wellness Program providing numerous resources for overall wellbeing