dyna-robotics

Model Evaluation Program Manager

Listed by employer · Checked

Posted within 60 days ·

  • onsite
  • FullTime
  • Operations
  • Redwood City, CA
Work arrangement
onsite
Employment
FullTime
Stated compensation
USD 140,000–160,000 1 YEAR
Stated sponsorship
Not stated
Apply on company website ↗
01 / A QUICK CHECK BEFORE YOU APPLY

Does the location policy fit?

Choose where you would work from. We’ll compare it with the posting’s stated requirements.

Choose the country you will work from. You can change this anytime.

Country and sponsorship changes take effect when you click Check eligibility.

Applying from Kenya

Not enough information in this posting — confirm with the employer

Checked for applicants in Kenya; employer sponsorship is not needed. Based on the source posting · not a hiring or work-authorization guarantee.

The posting does not specify whether applicants from your country are accepted.

Why this result?

3 still unclear

  • The posting does not specify whether applicants from your country are accepted.
  • Office attendance is required. Relocation and work permission have not been assessed.
  • Visa sponsorship is not specified.
Source: ashbyChecked Original posting ↗

Requirements

1 with evidence8 unclear or unstated
Remote policy
onsite
“Provider workplace type: onsite”
From employer posting metadata
Employment type
FullTime
Compensation
USD 140,000–160,000 1 YEAR
8 fields not stated in this posting
Employer sponsorship
Not specified
Geographic scope
Not specified
Degree requirement
Not specified
Security clearance
Not mentioned
Timezone / hours
Not specified
Citizenship requirement
Not mentioned
Minimum experience (years)
Not specified
Work authorization
Not specified

These details were not stated in the tracked posting. Confirm with the employer.

READ IT IN THE EMPLOYER’S OWN WORDS

Original job description

Reading view uses normalized posting text so exact evidence excerpts can be highlighted.

Dyna Robotics builds general-purpose robots powered by a proprietary embodied AI foundation model with top-in-industry generalization and real-world performance. Already deployed with customers across multiple industries, our robots do commercial-grade work in the physical world. Our team comes from Google DeepMind, Meta, and Cruise, and we're backed by CRV, First Round, and other leading investors. As a Model Evaluation Lead, you’ll lead the team responsible for understanding how our robot policies perform and whether new model versions deliver meaningful improvements. This is a full-time, on-site position in Redwood City.
Read full descriptionShow less description
What You’ll Do Team Management: Manage work flows for the evaluation team. Manage the team’s priorities. Establish clear ownership, provide regular coaching and feedback, and build onboarding and training practices that help team members execute evaluations and investigate issues independently. Research Partnership: Translate research questions into evaluation plans with clear hypotheses, success criteria, and deliverables. Partner with researchers to compare model checkpoints, assess new capabilities, and balance recurring evaluations with urgent experiments and available robot capacity. Evaluation Design: Develop task definitions, scoring rubrics, and test scenarios that reflect real operating conditions. Measure task success, output quality, cycle time, human interventions, and recovery behavior. Choose appropriate trial counts and account for variability when interpreting results. Repeatability & Traceability: Standardize setup, resets, scoring, and intervention rules. Ensure results are traceable to model versions, robot configurations, test conditions, logs, and videos. Coordinate with hardware and operations partners to resolve issues that affect evaluation quality. Failure Analysis: Review robot behavior, logs, and video with the team and researchers. Distinguish model failures from hardware, software, setup, or execution issues. Identify recurring failure patterns and recommend focused investigations, additional data collection, or annotation improvements. Regression Coverage & Reporting: Maintain consistent benchmarks and held-out test scenarios across tasks and conditions. Identify regressions, track progress across model versions, and communicate what improved, what remains unreliable, and where evidence is still limited. Provide clear recommendations that inform deployment decisions. Tools & Workflows: Use data analysis and lightweight scripting to improve reporting and reduce manual work. Partner with engineering on evaluation tooling, automation, and dashboards. Shorten the time from a new checkpoint to useful findings while preserving the quality of the evidence. What You’ll Bring The Experience: Experience in robotics, machine learning, applied research, or model evaluation, with direct ownership of evaluation design and analysis. Team Leadership: Demonstrated experience managing and developing a technical team, including setting priorities, giving feedback, and building effective working practices. Experimental Judgment: You can define success, control relevant variables, identify gaps in test coverage, and assess whether an apparent improvement is meaningful. Technical Fluency: Technical fluency to work directly with researchers on model checkpoints, generalization, data quality, and experimental tradeoffs. Data Analysis: Comfort using Python or similar tools to analyze results, inspect logs, and improve evaluation workflows. Communication & Ownership: You can turn an ambiguous request into an executable plan, surface problems early, and explain findings to both research and operations teams. Bonus Points For Physical-World ML Evaluation: Evaluating robot manipulation policies, embodied AI, autonomous systems, or other machine learning systems operating in the physical world. Evaluation Infrastructure: Building regression suites, benchmark datasets, failure taxonomies, or evaluation tools, and connecting findings to model development or deployment readiness. At Dyna Robotics, we build technology for the real world, which requires a team as diverse as the environments our robots inhabit. We are an equal opportunity employer committed to technical rigor and mutual respect. Don’t let a checklist stop you. Data shows that underrepresented groups often only apply if they meet 100% of the criteria. We value problem-solving and grit over keyword matching. If you’re passionate about robotics, no matter your discipline, we want to hear from you, even if you don't check every box.
Original formatting and links

Dyna Robotics builds general-purpose robots powered by a proprietary embodied AI foundation model with top-in-industry generalization and real-world performance. Already deployed with customers across multiple industries, our robots do commercial-grade work in the physical world. Our team comes from Google DeepMind, Meta, and Cruise, and we're backed by CRV, First Round, and other leading investors.


As a Model Evaluation Lead, you’ll lead the team responsible for understanding how our robot policies perform and whether new model versions deliver meaningful improvements.

This is a full-time, on-site position in Redwood City.

What You’ll Do

  • Team Management: Manage work flows for the evaluation team. Manage the team’s priorities. Establish clear ownership, provide regular coaching and feedback, and build onboarding and training practices that help team members execute evaluations and investigate issues independently.

  • Research Partnership: Translate research questions into evaluation plans with clear hypotheses, success criteria, and deliverables. Partner with researchers to compare model checkpoints, assess new capabilities, and balance recurring evaluations with urgent experiments and available robot capacity.

  • Evaluation Design: Develop task definitions, scoring rubrics, and test scenarios that reflect real operating conditions. Measure task success, output quality, cycle time, human interventions, and recovery behavior. Choose appropriate trial counts and account for variability when interpreting results.

  • Repeatability & Traceability: Standardize setup, resets, scoring, and intervention rules. Ensure results are traceable to model versions, robot configurations, test conditions, logs, and videos. Coordinate with hardware and operations partners to resolve issues that affect evaluation quality.

  • Failure Analysis: Review robot behavior, logs, and video with the team and researchers. Distinguish model failures from hardware, software, setup, or execution issues. Identify recurring failure patterns and recommend focused investigations, additional data collection, or annotation improvements.

  • Regression Coverage & Reporting: Maintain consistent benchmarks and held-out test scenarios across tasks and conditions. Identify regressions, track progress across model versions, and communicate what improved, what remains unreliable, and where evidence is still limited. Provide clear recommendations that inform deployment decisions.

  • Tools & Workflows: Use data analysis and lightweight scripting to improve reporting and reduce manual work. Partner with engineering on evaluation tooling, automation, and dashboards. Shorten the time from a new checkpoint to useful findings while preserving the quality of the evidence.

What You’ll Bring

  • The Experience: Experience in robotics, machine learning, applied research, or model evaluation, with direct ownership of evaluation design and analysis.

  • Team Leadership: Demonstrated experience managing and developing a technical team, including setting priorities, giving feedback, and building effective working practices.

  • Experimental Judgment: You can define success, control relevant variables, identify gaps in test coverage, and assess whether an apparent improvement is meaningful.

  • Technical Fluency: Technical fluency to work directly with researchers on model checkpoints, generalization, data quality, and experimental tradeoffs.

  • Data Analysis: Comfort using Python or similar tools to analyze results, inspect logs, and improve evaluation workflows.

  • Communication & Ownership: You can turn an ambiguous request into an executable plan, surface problems early, and explain findings to both research and operations teams.

Bonus Points For

  • Physical-World ML Evaluation: Evaluating robot manipulation policies, embodied AI, autonomous systems, or other machine learning systems operating in the physical world.

  • Evaluation Infrastructure: Building regression suites, benchmark datasets, failure taxonomies, or evaluation tools, and connecting findings to model development or deployment readiness.

At Dyna Robotics, we build technology for the real world, which requires a team as diverse as the environments our robots inhabit. We are an equal opportunity employer committed to technical rigor and mutual respect.

Don’t let a checklist stop you. Data shows that underrepresented groups often only apply if they meet 100% of the criteria. We value problem-solving and grit over keyword matching. If you’re passionate about robotics, no matter your discipline, we want to hear from you, even if you don't check every box.

No similar roles found yet — browse all dyna-robotics jobs.

Explore stated remote geography

These collections reflect applicant location statements, not authorization or citizenship.

THE JOB ELIGIBILITY CHECKER

Check a job