AI Evaluation Specialist – Italian (Remote)

About the role

This position focuses on assessing the quality of AI-generated responses in Italian, ensuring they are personalized, accurate, and helpful. You will work remotely, collaborating with a global team to improve large‑language‑model performance.

Key responsibilities

• Evaluate AI model responses for personalization quality, grounding, integration, and helpfulness.
• Design and execute multi‑turn prompts based on personal context to test AI capabilities.
• Analyze outputs for hallucinations, incorrect personalization, and flawed inferences.
• Perform side‑by‑side comparisons of model outputs to determine quality and effectiveness.
• Write clear rationales for response evaluations and rankings.
• Provide detailed feedback and annotations to improve model performance.
• Maintain strict data hygiene and accurate documentation of evaluations.

Required profile

• Strong proficiency in Italian with excellent reading and writing skills.
• Experience in data annotation, AI evaluation, content moderation, or similar roles.
• Analytical mindset capable of evaluating nuanced AI responses.
• Ability to design creative multi‑turn prompts based on personal context.
• Excellent written communication and attention to detail.
• Full‑time availability with flexibility to overlap with PST time zones.
• Bachelor’s degree or equivalent experience in a relevant analytical field.

Required skills

• Strong proficiency in Italian (reading and writing).

What we offer

• Hourly compensation ranging from $11.5 to $15.3.
• Flexible remote work schedule (20 hours per week).
• Opportunity to contribute to cutting‑edge AI personalization research.