AI Models & Local Execution (12)

12 free LLM evaluation and observability tools and 0 more — pricing tier and real capabilities on every entry. No ads, no sponsored placement.

AllLocal LLM RunnersModel Hosting & Inference APIsNotable Open ModelsSpeech & Transcription EnginesFine-Tuning & Training FrameworksLLM Eval & ObservabilityLeaderboards & BenchmarksRobotics & Embodied AI

Langfuse

LLM Eval & Observability
freemium
  • Fully open-source and self-hostable for free via Docker Compose/Kubernetes, not just a hosted SaaS
  • Hobby cloud plan is free with no credit card, 50k observability units/month
  • Combines tracing, prompt management, evals and datasets in one open platform
Visit

Braintrust

LLM Eval & Observability
freemium
  • Free Starter plan ships model credits and scored evals with no card required
  • Unlimited users/projects/datasets/playgrounds even on the free tier
  • Built-in playground for side-by-side prompt/model comparison against production traces
Visit

Promptfoo

LLM Eval & Observability
freemium
  • Open-source CLI/library for evaluating prompts, models and RAG pipelines side by side, runs in CI
  • Automated red-teaming to surface prompt injection, jailbreak and data-leak vulnerabilities
  • 300,000+ user community with an enterprise tier for teams needing hosted guardrails
Visit

W&B Weave

LLM Eval & Observability
freemium
  • Agent-native tracing model with sessions, steps, tools and sub-agents as first-class concepts (not generic spans)
  • Pre-built safety/quality scorers for toxicity, bias, PII and hallucination detection out of the box
  • Built on Weights & Biases' existing ML-experiment infrastructure, useful for teams already on W&B
Visit

Arize Phoenix

LLM Eval & Observability
free
  • Fully open-source, self-hostable with zero setup via `uvx` (pip/conda also supported)
  • Combines tracing, evals, datasets, experiments and prompt playground in one local-first tool
  • AI engineering agent (PXI) built in for automated troubleshooting of traces
Visit

Google Stax

LLM Eval & Observability
freeNew
  • Build evals with human raters or LLM autoraters on your own data
  • Compare models and prompts on quality, latency and token cost
  • Prebuilt and custom evaluators
Visit

DeepEval

LLM Eval & Observability
freeNew
  • Open-source LLM evaluation framework with pytest-style tests and 50+ metrics
  • RAG, agent and conversational metrics; red-teaming module
  • Apache-2.0; optional Confident AI cloud has a free tier
Visit

Opik (Comet)

LLM Eval & Observability
freemiumNew
  • Open-source LLM tracing, evaluation and prompt-optimization platform (Apache-2.0)
  • Free cloud plan with 25k spans a month, up to 10 team members and 60-day retention
  • Self-host with no limits
Visit

LangSmith

LLM Eval & Observability
freemiumNew
  • Tracing, evaluation datasets and prompt playground for LLM apps and agents
  • Developer plan is free with 5k base traces a month and one seat
  • Framework-agnostic, works beyond LangChain
Visit

EleutherAI LM Evaluation Harness

LLM Eval & Observability
freeNew
  • Standard open framework for running hundreds of benchmarks (MMLU, GSM8K and more) against any LLM
  • Backs the Hugging Face Open LLM evaluations
  • MIT licensed CLI and Python library
Visit

OpenLLMetry (Traceloop)

LLM Eval & Observability
freeNew
  • OpenTelemetry-based instrumentation for LLM calls, vector DBs and frameworks
  • Ships traces to Datadog, Grafana, Honeycomb or any OTel backend
  • Apache-2.0
Visit

Laminar

LLM Eval & Observability
freeNew
  • Open-source observability and evals platform for AI agents (Apache-2.0)
  • Traces, session replay for browser agents and evaluations
  • Self-hostable
Visit

LLM Eval & Observability — straight answers

What are the best free LLM evaluation and observability tools?

Flocci AI Tools lists 12 LLM evaluation and observability tools, ordered free tiers first: Arize Phoenix, Google Stax, Laminar, DeepEval and EleutherAI LM Evaluation Harness. All of them are free or have a free tier. Every entry states what it uniquely does, not a score.

Are there completely free LLM evaluation and observability tools?

Yes — 6 of these are marked fully free with no paid plan attached: Arize Phoenix, Google Stax, Laminar, DeepEval and EleutherAI LM Evaluation Harness. The rest are freemium, trial or paid, and each row states which.

What should I look for in LLM evaluation and observability tools?

Start with what you need to do: trace, score and debug what your AI app actually did. Then check the pricing tier, because "free" here means no paid plan at all while "freemium" means a free tier under paid plans. Every tool on this page lists both.

See the full list →

Head-to-head comparisons

Arize Phoenix vs Google StaxArize Phoenix vs LaminarArize Phoenix vs DeepEvalArize Phoenix vs EleutherAI LM Evaluation HarnessArize Phoenix vs OpenLLMetry (Traceloop)Google Stax vs LaminarDeepEval vs Google StaxEleutherAI LM Evaluation Harness vs Google Stax

Free alternatives

Langfuse alternativesBraintrust alternativesPromptfoo alternativesW&B Weave alternativesArize Phoenix alternativesGoogle Stax alternativesDeepEval alternativesOpik (Comet) alternatives

Looking for a ranked shortlist instead? See the best free LLM evaluation and observability tools.