LLM Language Evaluation Specialist

About the role

We are looking for Albanian language specialists to participate in a long‑term LLM Language Evaluation Project focused on assessing the quality and accuracy of AI‑generated content. The engagement will run for at least three months, beginning with a short paid trial and training phase before moving to full‑scale production.

Key responsibilities

• Evaluate AI/LLM‑generated Albanian text against detailed project guidelines, checking for accuracy, naturalness and linguistic appropriateness.
• Identify, classify and document language‑related issues such as mistranslations, missing content, terminology errors, spelling, grammar, punctuation and other readability problems.
• Spot abnormal cases including non‑Albanian content, language identification errors, duplicate entries or material that must be rejected.
• Apply evaluation criteria consistently and deliver structured results while maintaining high attention to detail.

Required profile

• High proficiency in Albanian.
• Previous experience with Large Language Models, AI evaluation, data annotation, localization or translation projects.
• Strong understanding of Slovak grammar, vocabulary and terminology (as required by the project).
• Ability to follow detailed evaluation guidelines and identify subtle language errors.
• Comfortable working with structured digital documents and online tools.