Data Annotator – AI Quality Analyst (Italian) – Remote

About the role

We are looking for a detail‑oriented Data Annotator to evaluate and improve the personalization quality of large language model (LLM) responses in Italian. This remote, hourly‑contract position involves testing AI capabilities with multi‑turn prompts and providing clear feedback to enhance model performance.

Key responsibilities

• Assess AI model outputs for grounding, integration, helpfulness, and personalization quality.
• Design and execute multi‑turn prompts based on personal context to probe model behavior.
• Identify hallucinations, incorrect personalization, and flawed inferences in responses.
• Conduct side‑by‑side comparisons of model outputs to determine relative quality.
• Write structured rationales and rankings for each evaluation.
• Provide detailed annotations and feedback to guide model improvements.
• Maintain strict data hygiene and accurate documentation of all evaluations.

Required profile

• Native‑level proficiency in Italian with excellent reading and writing skills.
• Experience in data annotation, AI evaluation, content moderation, or similar analytical roles.
• Strong analytical thinking with the ability to evaluate nuanced AI responses.
• Creativity in designing multi‑turn prompts based on personal context.
• Excellent written communication and meticulous attention to detail.
• Full‑time availability with flexibility to overlap with PST working hours.
• Bachelor’s degree or equivalent experience in a relevant analytical field.

Required skills

What we offer

• Hourly compensation ranging from $11 to $15.
• Fully remote work arrangement.
• Opportunity to contribute to cutting‑edge AI personalization research.