RL environments · Canada
Simulated banking, insurance, public-service and bilingual tasks, with rewards written and verified by Canadian professionals. Built to run in your trainer, or in ours for evaluation.
A task, the tools a model may use, and a verifier that scores what the model did. The model attempts the task many times. Each attempt is scored. The scores train the next version of the model, or, for evaluation, describe the version you have.
Human Data writes and verifies the nine components on the left: realistic Canadian tasks, the tools and state that make them real, and the rewards that decide what counts as correct. The loop on the right runs in your training stack. For evaluation, it runs in ours.
Every RL training system has the same five stages. Frameworks differ in how many they cover. Human Data supplies the three stages that need domain experts and hands off to whichever rollout and trainer you run. For evaluation engagements we also run the rollouts.
Framework coverage as mapped in the Hugging Face guide to RL environments (May 2026). Select a chip to isolate one row. Environments are delivered as standard Python packages and can target the OpenEnv, Verifiers, SkyRL or NeMo Gym interfaces.
Each environment is built from real Canadian products, rules and documents, in English and Canadian French.
Mortgage renewals, registered-account transfers (TFSA, RRSP, FHSA), disputes and KYC onboarding against Canadian product rules. Rewards check the account, the amount, the consent and the disclosure.
Claims intake, coverage questions, renewals and underwriting triage built on Quebec and common-law policy wording. Rewards check coverage logic, exclusions and escalation.
Benefit applications, permit workflows and correspondence over public-service documents in both official languages. Rewards check eligibility rules, language requirements and hand-off to a person.
Payments, bill pay, card disputes and member service for smaller institutions, with the vendor interfaces they actually use. Rewards check payee, amount, timing and member consent.
Every environment runs in English and Canadian French with Quebec terminology and register. Rewards check meaning, terminology and that French clients are served in French.
An agent that says “done” is not evidence that the right thing happened. Our environments record every action and compare it with the instruction, the policy and the client’s situation. Canadian evaluators adjudicate the gap, in English and French.
Move $2,000 into my TFSA before Friday.
Done. $2,000 moved.
Published results for open-weight models you can run yourself. Our Canadian environments add the products, rules and French that these benchmarks do not cover.
Text agents resolving banking customer-service tasks over a knowledge base of about 700 documents, with a simulated customer and a policy the agent must follow.
| Model | pass@1 | pass@1 |
|---|---|---|
| Kimi K3Open weightsMoonshot AI · modified MIT · all tools, extended reasoning | 37.10% | |
| GLM-5.2Open weightsZ.ai · MIT licence · all tools, extended reasoning | 37.10% | |
| InklingOpen weightsThinking Machines · Apache 2.0 · all tools, extended reasoning | 25% | |
| GLM-5Open weightsZ.ai · MIT licence · embedding retrieval, no extended reasoning | 9.80% | |
| Qwen3.5-397B-A17BOpen weightsAlibaba · Apache 2.0 · embedding retrieval, no extended reasoning | 9.80% |
Four suites including banking. Published open-weight result: Llama 3 70B completes 34.02% of tasks, 18.28% under attack; 25.60% of injections succeed.
ETH Zurich · NeurIPS 202411 harm categories including fraud; measures whether agents comply with harmful multi-step requests.
Gray Swan AI · UK AI Security InstituteDocument-grounded cases written with practising underwriters and claims handlers, scored pass@1. Leaderboard opening 2026.
Huzzle LabsFrom a professional’s description of the work to a package your trainer can run.
Practising Canadian professionals write scenarios from real work: the products, the documents, the edge cases and the French version. Each task has a gold outcome.
Tools, state and documents become a sandbox: a mock core-banking interface, a policy database, a case file. Nothing touches a production system.
Deterministic checks where the answer is exact (account, amount, date). Expert rubrics where judgment is needed. Every rubric is calibrated on a shared sample before use.
Two experts review each task; disagreements go to a senior adjudicator. Environments ship as Python packages in the interface your trainer expects.
Environments run as containers. The task and the reward are what need experts; running thousands of sessions is a scheduling problem the ecosystem has already solved.
All at 95% or higher session success. Source: OpenEnv scaling benchmark, cited in the Hugging Face guide to RL environments (May 2026).
Tell us the domain, the trainer you run and the behaviour you need to measure.