Looking for a free alternative to LangSmith? Here are 9 llm eval & observability worth trying — 9 with a free tier. LangSmith itself is free tier + paid.
| Tool | LangSmith (you searched) | Arize Phoenix | Google Stax | DeepEval |
|---|---|---|---|---|
| Pricing | freemium | free | free | free |
| Best for | Tracing, evaluation datasets and prompt playground for LLM apps and agents | Fully open-source, self-hostable with zero setup via `uvx` (pip/conda also supported) | Build evals with human raters or LLM autoraters on your own data | Open-source LLM evaluation framework with pytest-style tests and 50+ metrics |
| Visit | Open ↗ | Open ↗ | Open ↗ | Open ↗ |
Arize Phoenix is the closest free alternative to LangSmith: same LLM Eval & Observability sub-category, and it is free. Fully open-source, self-hostable with zero setup via `uvx` (pip/conda also supported)
Yes — 9 of the 9 alternatives listed here are free or freemium: Arize Phoenix, Google Stax, DeepEval, EleutherAI LM Evaluation Harness, OpenLLMetry (Traceloop). None of them are paid-only.
LangSmith is free tier + paid. There is a free tier you can work in, with paid plans above it.
They are the other tools in LLM Eval & Observability, then the rest of AI Models & Local Execution, ordered free tiers first and capped at 9. There is no sponsorship and no paid placement in that ordering — only the pricing tier decides.
✦ WhyA LangSmith alternative — Fully open-source, self-hostable with zero setup via `uvx` (pip/conda also supported).
✦ WhyA LangSmith alternative — Build evals with human raters or LLM autoraters on your own data.
✦ WhyA LangSmith alternative — Open-source LLM evaluation framework with pytest-style tests and 50+ metrics.
✦ WhyA LangSmith alternative — Standard open framework for running hundreds of benchmarks (MMLU, GSM8K and more) against any LLM.
✦ WhyA LangSmith alternative — OpenTelemetry-based instrumentation for LLM calls, vector DBs and frameworks.
✦ WhyA LangSmith alternative — Open-source observability and evals platform for AI agents (Apache-2.0).
✦ WhyA LangSmith alternative — Fully open-source and self-hostable for free via Docker Compose/Kubernetes, not just a hosted SaaS.
✦ WhyA LangSmith alternative — Free Starter plan ships model credits and scored evals with no card required.
✦ WhyA LangSmith alternative — Open-source CLI/library for evaluating prompts, models and RAG pipelines side by side, runs in CI.