Python Expert - AI Evaluator - (Freelance, Remote, 20+ openings)

Role: Engineering

Rate: $10.00–$65.00 USD/hr

Skills: Data Annotation, Python

Project Description

This project involves reviewing Al-generated responses, evaluating technical accuracy and

best practices, and providing expert feedback that trains Al systems to write better code.

You'll work with various programming languages and coding challenges, ensuring Al outputs

are functional, efficient, and follow industry standards.

Compensation

Compensation is competitive and varies based on education level, institution, and prior experience.

Hiring Profile

• • BS, MS, or PhD with significant focus on Computer Science (Required).
• • Expert in Python, with a strong grasp of algorithms, data structures, and debugging
• practices.
• • Expert-level English proficiency with excellent writing skills.
• • Expert at fact-checking information using public sources across multiple domains.
• • Significant experience using large language models (LLMs).
• • Strong analytical and structured thinking skills.
• • Ability to explain complex programming concepts clearly and accurately.
• • Prior experience with RLHF annotation (preferred).
• • Prior Software Engineering work experience (strongly preferred).
• • Expertise in additional programming languages (nice to have): C#, Java, SQL, C++,
• TypeScript, PHP, C, Go, Bash, PowerShell, Rust, R.

Application & Onboarding Process

• Apply via partner site - Braintrust will send you an email with detailed instructions.
• Qualification exam (1–2 hours): evaluates your role-specific expertise. Compensation provided upon passing.
• Project exam (1–2 hours): evaluates your adherence to project guidelines and ability to produce math-specific deliverables. Compensation provided upon passing.

Commitment

• We expect a minimum of 17–20 hours per week, however considering the project scope and timelines the preferred and recommended commitment would be 8 hours/day.

#LI-Remote