How to Test an Arabic AI Assistant Before Rolling It Out to Staff

Why a few trial questions are not enough

Most internal assistants are tried before launch with questions written by the project team, so the answers look convincing. Real employees, however, ask in their own dialect, abbreviate, write dates in different formats and ask about documents they are not entitled to see.

NIST’s Generative AI Profile, published in 2024, warns against extrapolating system performance from narrow, non-systematic assessments, and calls for performance to be demonstrated in conditions similar to the deployment setting, with the expected context of use documented. ^[1] The profile also notes that language models may perform less well in non-English languages or certain dialects, and that they can present false information with complete confidence. ^[1] Both are direct risks for any assistant that answers in Arabic.

What goes into the test set?

Start by gathering real questions: service desk logs, staff emails to HR, and recurring queries about policies. Then add what a casual trial never surfaces.

For each question, write down three things: the reference answer approved by a specialist, the document and paragraph it rests on, and the expected behaviour if an answer is not possible, such as asking for clarification or referring the user to the right team. Evaluators such as response completeness measure whether a reply contains the key information compared with the reference answer, so completeness cannot be judged without a reference answer written in advance. ^[4]

The questions and reference answers should be written by specialists from the departments that own the policies, not by the technical team alone, because they know which questions staff usually get wrong. Weight the categories by the impact of an error, so that allowances, leave and permissions get a larger share than general information.

Keep the acceptance set separate from the examples used to tune prompts or configure the assistant. If the team adjusts settings until the acceptance questions themselves pass, it has measured its ability to memorise the questions, not the quality of the assistant.

How should each answer be scored?

Do not give each answer a single score. Split the evaluation into independent dimensions, because an answer can be correct in substance but drawn from a document the user may not see, or grounded in the right source but incomplete.

Evaluation platforms offer automated metrics that help with triage, including groundedness, which estimates how well the retrieved context supports the answer, and relevance to the question. ^[4] These save time on large sets but do not replace a specialist’s judgement. NIST recommends comparing outputs with reference answers using both human oversight and automated evaluation, and reviewing the sources and citations the system provides. ^[1]

Attribution is tested by hand: open the cited document, match the paragraph and confirm it is from the current version. A link in the answer does not mean the information is actually there.

Can the user see what they should not?

An assistant that answers from organisational documents retrieves passages and sends them to the model. If document permissions are not enforced at retrieval, the answer may surface information from a file the user is not allowed to open. ^[3] Microsoft notes that trimming results by permission inside the search pipeline itself removes the need for custom permission code in the application, and is faster than loading larger result sets and trimming them there. ^[3]

Create test accounts that represent different roles: an ordinary employee, a department head and an employee from another department. Ask each account the same question and check that the answer reveals nothing beyond its access, not even as a summary or a hint. Also test what happens after access is withdrawn, because permission changes appear in search results only once the permission metadata has synced to the index. ^[3]

Testing untrusted input

The document itself can carry risk. Indirect prompt injection happens when the model reads external content, such as a file or web page, containing instructions that alter its behaviour. ^[2] OWASP stresses that retrieval-augmented generation and fine-tuning do not fully remove this risk, and that attackers may use multiple languages or encodings to slip past filters. ^[2]

Include documents prepared for this purpose in the test environment, containing text in Arabic and English that tells the assistant to abandon its task or reveal other content. The expected result is that the assistant carries on with its original task and the application blocks any action or disclosure beyond the user’s permissions. NIST lists red-teaming against attacks such as prompt injection among its measurement actions. ^[1]

Hypothetical example: a travel allowance question

A hypothetical organisation has built an assistant that answers questions on HR policies, and has approved a new travel policy taking effect at the start of next month. The team designed three cases:

• A question with a clear date for a trip after the effective date: the expected answer applies the new policy and states when it takes effect.
• A question with no date: the assistant should ask when the trip is, or set out both the old and new rules with the changeover date, rather than silently choosing one.
• A question from an employee without access to the senior management allowance table: neither the attachment nor its figures should appear in the answer.

An HR specialist reviewed all three answers, and the results were logged with the model version and documents used, so that the same cases can be rerun after every update.

How to read the results and decide on launch

Do not base the launch decision on the average pass rate alone. Divide errors into two categories:

1. Critical errors: disclosing content beyond the user’s permissions, following injected instructions, or giving a wrong answer on a financial or regulatory matter. These are fixed before launch, however rare they are.
2. Quality errors: an incomplete answer, weak wording or an imprecise referral. These are measured against a threshold the service owner sets in advance.

Record the limits of acceptable use in the launch decision: which topics the assistant answers, and which it refers to a member of staff.

To make results comparable from one round to the next, log for each test case: the question ID, the model version and settings, the version of the indexed documents, the role of the account that asked, the result on each dimension, and the reviewer’s name and review date. Without this log, after an update you cannot tell whether the assistant improved or the questions changed.

When should you retest?

Keep the same test set as a regression suite and rerun it when the model or its settings change, when documents are updated or new sources added, and when permissions are modified. NIST recommends re-evaluating risks when a model is adapted to a new domain and checking regularly that security measures remain effective. ^[1]

Applying this in Gulf workplaces

Involve specialists from different sites to cover local terminology, since the name of an allowance or type of leave may differ from one organisation to another. Include the Hijri and Gregorian date formats and the currencies your organisation deals in.

Arabic has features that affect retrieval before they affect the answer: variant spellings of the hamza, taa marbuta written as haa, alif maqsura written as yaa, and English terms transliterated into Arabic script. Write the same question in more than one spelling and check that the assistant reaches the right document each time. If retrieval fails, no amount of model quality will rescue the answer.

Do not put real employee data in test examples unless the environment is approved for it and appropriate controls are in place, and comply with the personal data protection law in force in your country, such as Saudi Arabia’s Personal Data Protection Law or the UAE’s Federal Decree-Law No. 45 of 2021 on the Protection of Personal Data.

Conclusion

Testing an Arabic assistant is a project in its own right: questions from real work, reference answers, separate scoring dimensions, and tests for permissions and injected input. Anything tested once needs testing again whenever the model, the documents or the users change.