Quality Analyst (Japanese)

Position: LLM AI Quality Analyst (Personalization) Japanese

Type: Short-Term Contract

Location: Remote (USA & Japan)

Commitment: Part-time (20 40 hours/week, overlap with PST required)

Compensation: $15 per hour

Engagement Length: 1 month

Start Date: Immediate

Role Responsibilities

• Design and execute multi-turn conversational prompts to evaluate AI personalization quality
• Assess how AI models use personal context from Gemini, Gmail, Google Search, and YouTube activity
• Evaluate responses for grounding, integration quality, helpfulness, and natural personalization
• Compare and rank side-by-side model outputs with detailed written rationales
• Identify hallucinations, weak inferences, and over-personalized responses
• Extract and verify debug information to confirm correct data-source utilization
• Maintain high-quality annotations and structured evaluation documentation
• Follow strict data hygiene and evaluation protocols

Requirements

• Strong Japanese reading and writing proficiency
• Experience in AI quality evaluation, data annotation, content moderation, or related analytical work
• Strong analytical thinking with ability to evaluate nuanced AI responses
• Ability to create creative multi-turn prompts based on personal context
• Excellent written communication and structured feedback skills
• Comfortable using a primary personal Google account for evaluation purposes
• Self-motivated and able to work independently in a remote environment
• Reliable desktop/laptop setup with stable internet connection

Application Process

• Fill out the application form
• Complete the ICF
• Assessment
• Pre-onboarding discussion for shortlisted candidates