12 Levels — Build, Train, Certify

Build Agent
Harnesses.
Level by Level.

A training gym where developers build, train, and certify their own AI agent harnesses across 12 progressive levels. Each level adds a capability — from basic chat to production deployment. Auto-evaluated. Trajectory-backed. Production-ready.

Python-native. Runs locally. Deploys on ICP. Your harness, your rules.

See All 12 Levels → View on GitHub
// ARLI Academy — meta-harness CLI
$ arli-academy train level_01
Loading level spec...
Exercise: Basic Chat Loop
Evaluator: loaded ✓
Running 30 test scenarios...
████████████████████ 28/30 passed
Score: 93/100 — PASS ✓
Coaching feedback:
• Handle empty messages gracefully
• Add timeout to provider calls
→ Level 02 unlocked: Tool Calling
The Gym — June 2026
12
Levels Total
12
Built & Tested
90
Tests Passing
30
Tests Per Level
3
Blocks (1–4, 5–8, 9–12)
The White Space

Nobody Sells Harness Training

The market has frameworks (LangChain, CrewAI — opinionated monoliths), point solutions (iii workers — composable, no curriculum), and eval sandboxes (Daytona, E2B — infrastructure, no pedagogy). Nobody combines curated curriculum × sandboxed execution × automated evaluation × trajectory analytics.

The Problem Today

AI engineers learn by reading docs and copy-pasting prompts. They pay $20–200/month for tools they don't understand, their data goes to third-party servers, and they can't automate workflows beyond single-turn chat. No structured path exists to go from "I use ChatGPT" to "I own a production harness."

No curriculum for harness engineering
No automated evaluation of agent quality
No certification standard
No trajectory data for improvement
No marketplace for trained harnesses

What The Gym Delivers

Structured curriculum. 12 levels, each with a real exercise, automated grading, and coaching feedback.

Trajectory data. Every submission recorded — replay, audit, improve. Becomes a proprietary dataset for benchmarking.

Certification. Graduate with a verifiable, sellable harness.

Marketplace. Sell your certified harness or buy one pre-trained for your industry.

"Build your harness. We'll evaluate it. Sell it when you're done."
— Harness Training Gym Thesis, June 2026
Process

How a Level Works

From reading the brief to unlocking the next level — the complete training loop.

01
Read the Level Brief
Each level starts with a spec: what capability to build, technical constraints, and the evaluation rubric. You know exactly what "pass" looks like before writing a single line of code. The spec includes a gold-standard reference implementation for comparison.
level_01_basic_chat/__init__.py — Spec + Evaluator + Reference
02
Build in Your Sandbox
Write your implementation in Python — the same language as the Hermes agent harness. Run locally with the CLI. Iterate until you're confident. Each level is self-contained; no API keys needed for core levels.
arli-academy train level_01 — Run, test, iterate locally
03
Submit for Evaluation
Push your code. The meta-harness runs 10–50 test scenarios against your implementation — testing edge cases, error handling, and performance. Every run is recorded as a trajectory for replay and audit.
arli-academy eval level_01 — 30 scenarios, auto-graded
04
Get Coaching Feedback
Pass or fail, you get detailed feedback. The LLM-powered grader enriches raw scores with coaching comments: what you did well, what broke, and how to fix it. Re-submit until you pass. Trajectory comparison shows your progress.
Grader: score + coaching_feedback + trajectory_diff
05
Advance to Next Level
Pass → unlock Level N+1. Complete all 12 → certified harness. Graduate with a production-ready agent you can sell on the marketplace or deploy to your own infrastructure on the Internet Computer.
Status: Passed → Level 02 unlocked: Tool Calling
Curriculum

12 Levels. One Harness.

Three blocks of four. Each block sold separately. Graduate with all 12 and get certified.

Block 1 — Foundations

Level 01
Basic Chat
Single-turn LLM conversation with provider abstraction.
Built ✓
Level 02
Tool Calling
Register, parse, execute tools. Return results to LLM.
Built ✓
Level 03
Memory
SQLite persistence with FTS5 search across sessions.
Built ✓
Level 04
ReAct Loop
Reason → Act → Observe cycle with multi-turn tools.
Built ✓

Block 2 — Intermediate

Level 05
Multi-Step
Chained tool execution with dependency resolution.
Built ✓
Level 06
Swarm
Multi-agent delegation with subagent spawning.
Built ✓
Level 07
RAG
Retrieval-augmented generation with vector search.
Built ✓
Level 08
Evaluation
Auto-eval harness: test suites, scoring, trajectory replay.
Built ✓

Block 3 — Advanced

Level 09
Human-in-Loop
Approval gates, interrupts, human override patterns.
Built ✓
Level 10
Optimization
Token compression, context mgmt, cost tracking.
Built ✓
Level 11
Security
Permission scoping, toolset restrictions, audit log.
Built ✓
Level 12
Production
ICP canister deploy, certified assets, CI/CD pipeline.
Built ✓
Stack

Three-Layer Architecture

The training gym sits on top of the student's harness, which is built on top of the reference harness (Hermes). Each layer is independently testable.

🎯

Layer 3: Training Gym

Curriculum engine (12 levels), exercise sandbox per student, evaluation harness with auto-grading, trajectory store with replay, certification and marketplace.

🔧

Layer 2: Your Harness

What you build — level by level. Starts as a simple chat loop at Level 1. Gains tool calling, memory, ReAct, swarm, RAG, evaluation, security, and production deployment by Level 12.

📦

Layer 1: Hermes

Reference harness — 20+ providers, tool registry, SQLite sessions with FTS5, context compression, subagent delegation, multi-platform gateway, plugins, cron scheduler.

June 17, 2026 — Today's Build

What We Built Today

The core meta-harness — a working evaluation pipeline with 2 production-ready levels, 10 scaffolded, a full CLI, and an LLM-powered grader.

ARLI Academy Core

A Python-based meta-harness that evaluates student agent implementations. Self-contained — no external API keys needed for core levels. Runs locally, deploys on ICP.

Python ICP CLI Hermes SDK

Trainer Modules

harness.py — meta-harness: loads levels, runs evaluations, produces scores.

grader.py — LLM-powered feedback enrichment.

runner.py — sandboxed execution with benchmarks.

cli.py — train, eval, status commands.

Levels 1–12: All Built ✓

90/90 tests passing across all 12 levels. Full spec + evaluator + gold-standard reference implementation for each level.

Chat → Tools → Memory → ReAct → Multi-Step → Swarm → RAG → Evaluation → Human-in-the-Loop → Optimization → Security → Production Deploy. Ready for trainees.

Production

Ready for Prime Time

The meta-harness is complete — auto-grading, trajectory recording, coaching feedback. Next: LLM grader enrichment, web interface, student sandbox, certification marketplace.

Ready to Build Your Harness?

Start at Level 01. Pass all 12. Graduate with a certified, production-ready agent harness you can sell or deploy.

View on GitHub → Explore All 12 Levels

ENSO Gym · Built June 17, 2026 · arliwork/arli-academy · Internet Computer