Senior Software Engineer – LLM Evaluation

About Turing

Based in San Francisco, California, Turing is a research accelerator for frontier AI labs and a partner for global enterprises deploying advanced AI systems. Turing accelerates frontier research with high-quality data, advanced training pipelines, and researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents. It also helps enterprises turn AI into proprietary intelligence with reliable systems and measurable impact.

Role overview

As a Software Engineering evaluator, create datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. Curate code examples, provide precise solutions, and make corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go. Evaluate and refine AI-generated code for efficiency, scalability, and reliability, and work with cross-functional teams to enhance enterprise-level AI-driven coding solutions.

Day-to-day responsibilities

• Curate code examples, build solutions, and correct code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go
• Evaluate and refine AI-generated code for efficiency, scalability, and reliability
• Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks
• Build agents that verify code quality and identify error patterns
• Form hypotheses across the software engineering cycle: prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, and operational maintenance; evaluate model capabilities at each stage
• Design verification mechanisms that automatically verify solutions to software engineering tasks

Requirements

• Several years of software engineering experience, including 2+ years of continuous full-time experience at a top-tier product company (examples: Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research)
• Strong expertise building full-stack applications and deploying scalable, production-grade software using modern languages and tools
• Deep understanding of software architecture, design, development, debugging, and code quality/review assessment
• Excellent oral and written communication skills for clear, structured evaluation rationales

Required skills

JavaScript and Python.

Offer details

• Remote contractor
• Flexible engagement: minimum 10 hours/week, up to 40 hours/week
• Partial overlap with Pacific time required
• No medical benefits or paid leave
• Duration: one month, with potential extensions based on performance and fit
• The supplied listing states a start next week; confirm the current start date with Turing
• Advertised referral bonus: $500 (a referral incentive, not the candidate salary)
• Candidate compensation was not specified in the supplied listing