Article
Evaluating AI Agents: Metrics, Traces, and Safety
A practical framework for agent evaluation
Agent evaluation should be systematic and repeatable. We detail task success metrics, trace-based debugging, and safety policies for agents using open-source LLMs.
Evaluation Suite
- Task success & quality scores
- Tool error analysis
- Latency & cost dashboards
- Safety policy violations
More technical articles
Building Systematic AI Agents with Open-Source LLMs
A practical blueprint using Llama, Mistral, and LangChain
Productionizing Agent Workflows: LangChain, AutoGen, and Llama
From notebooks to robust services
Big Prometheus at Scale: Pros, Cons, and a MoSCoW Map of Observability Tools
TenthMatrix rewrite of the Big Prometheus problem space: Thanos, Cortex/Mimir, VictoriaMetrics, M3/Chronosphere — plus MoSCoW across OTEL, Grafana, SaaS APM and more