Braintrust

Founder backgrounds

• ▶1:54

  2d ago·6 views

  Product updates: Scatterplots and Java auto-instrumentation

  Braintrust ↗

• Evals

  Evals is Braintrust's core product for systematically measuring LLM application quality. Engineers define datasets of inputs and expected outputs, then run experiments across prompts, models, and code paths with built-in or custom scoring functions — including LLM-as-a-judge graders. Results are diffed across runs so teams can see exactly which prompt change moved which metric, on which examples. Evals turns ad-hoc 'vibe testing' into a reproducible, CI-friendly workflow that gates production deploys on quality.

• Playground

  Playground is Braintrust's prompt engineering and experimentation environment. Engineers and PMs can iterate on prompts side-by-side across multiple models and providers, replay them against eval datasets in real time, and immediately compare outputs and scores. Saved prompts version automatically and can be promoted to production via the SDK without code changes. The Playground tightens the loop between prompt iteration and rigorous evaluation, which is where most AI-product velocity is gained or lost.

• Logs and Observability

  Braintrust's logging captures full LLM traces — inputs, outputs, intermediate tool calls, latency, cost, and user feedback — from production AI applications. Traces stream into the same workspace as evals, so engineers can spot regressions, cluster failures, and add real production examples directly into eval datasets with one click. This closes the loop from production back to development, letting teams continuously improve quality on the inputs their users actually send.