How can organisations ensure that AI-powered chatbots deliver correct, fair, and coherent information to employees?

This question is at the heart of a recent assessment, conducted with the proprietary MIRAI Toolbox on a chatbot internally deployed within the HR Office of a large financial Institution. MIRAI was asked to perform quantitative tests to check for:

1. Correctness: how much the answers by the chatbot were correct with respect to the ground truth of the knowledge base;
2. Fairness: how fair was the chatbot when comparing answers to questions falling within the same scenario, but asked from different user profiles;
3. Coherence: how coherent was the chatbot when comparing answers to the same questions asked by different profiles within the company.

To evaluate these metrics, MIRAI produced a synthetic dataset of interactions, creating user personas that differ by position within the company, as well as by socioeconomic and educational factors, and including them in different interaction scenarios.

This quantitative assessment was then combined with a qualitative analysis of the use of the chatbot within the organization performed by Immanence SRL.

Key Findings by the MIRAI Toolbox

• Sub-optimal Correctness: the chatbot provided suboptimal performance for information correctness. In particular requests for remote working and parental leave were subject to incorrect or imprecise answers, or coming from sources other than the knowledge base. In 2.6% of tests for fairness the chatbot produced hallucinations.
• Low Immediate Risk of Fairness, but Gaps Remain: The risk of severe violations of fundamental rights by the chatbot was assessed to be low. However, notable gaps have been identified, such as the absence of explicit mechanisms to identify and manage potential impacts on privacy, non-discrimination, and freedom of information. In 4.54% of tests for fairness the chatbot produced hallucinations.
• Satisfying Coherence standards: The number of violations for our coherence checks have been limited. Within several categories of employees, junior are slightly disadvantaged with respect to senior ones, with answers more complete and precise for the latter profiles. No hallucinations were detected during this test.

Methodology

MIRAI’s assessment was based on quantitative testing using our Toolbox.

Modules:

• Quantitative Evaluation: The text-based interactions between empoyees and the chatbot are evaluated with a number of semantic metrics
• Fairness Analysis: Identifying and selecting protected attributes, the model is investigated for discrimination and distortions.
• Correctness Analysis: Correctness is evaluated by measuring distances of the answers from the ground truth.
• Coherence Analysis: Coherence is evaluated by measuring distances of the answers for different profiles, with respect to the neutral-case scenario.
• Risk Measurement: Test results are aggregated to evaluate the risk of equity violations.

Quantitative Results

• Correctness: The chatbot’s risk of providing unsatisfactory answers is high (QRisk: 76.1%–82.7%, depending on threshold). Remote work and parental leave topics are particularly problematic, with frequent inaccuracies and occasional hallucinations (2.7% of cases).
• Fairness: Risk is critical when considering gender and disability (QRisk: 100%), and high for background (89.5%) and role (67.6%). Women in junior roles and non-disabled individuals generally receive more equitable responses, while senior men and people with disabilities are more likely to experience unfairness.
• Coherence: The chatbot performs better here, with a lower risk (QRisk: 26.6%) and no hallucinations detected.

Overall Risk

When aggregating all results, the overall risk (QRisk) of the chatbot providing incorrect, unfair, or incoherent information is high. However, after adjusting for the specific risk profile of the project, the effective risk is estimated at 33%.

Recommendations

• Continuous Monitoring: Implement ongoing checks for privacy, non-discrimination, and information rights impacts.
• Human Oversight: Maintain human-in-the-loop processes for sensitive queries.
• Targeted Improvements: Focus on improving accuracy in remote work and parental leave topics, and address fairness gaps for senior men and people with disabilities.
• Transparent Reporting: Regularly publish risk metrics and fairness assessments to build trust and accountability.

—

Conclusion:
Our assessment demonstrates the value of rigorous, multidimensional evaluation for AI systems in the workplace. By combining technical analysis with user feedback, organisations can identify both strengths and areas for improvement—ensuring that AI tools support, rather than undermine, fairness and fundamental rights.

—