A Windows desktop application for evaluating local GGUF models served by llama.cpp. It scans a models folder, starts or attaches to a llama-server on 127.0.0.1:8080, and runs configurable test-case suites covering logic, code, format compliance, instruction following, generation speed and long-context handling. Each answer is graded by an automatic check - substring, regex, JSON schema, code execution, LLM-as-judge or embedding similarity - with skipped checks kept out of the score instead of counted as failures. Runs are stored in a SQLite history with per-case drill-down and side-by-side comparison, and can be exported as an HTML report or printed. Also includes a local agent mode that uses the tested model as a subagent, and reports prefill and generation speed, time to first token, model memory usage and layer placement from the server log.
Publisher’s description
What is LLM Test Bench?
LLM Test Bench is an open-source ai assistants & llm program for Windows from aleksandrreva-eng.
Installation and requirements
Official installers exist for 64-bit as MSI. It can be installed for the current user only, without administrator rights. The installer supports unattended (silent) installation. An MSI package is available for deployment with Group Policy, Intune or Configuration Manager.
Pros / Cons
Pros
- Open source: the code can be inspected and built by anyone
- MSI package for managed deployment
- Documented silent install
- Installs without administrator rights
- Updated in the last two months
Cons
- No native ARM64 build
In short: LLM Test Bench is an open-source ai assistants & llm tool for 64-bit PCs. Administrators can deploy it unattended.
Is LLM Test Bench safe to download?
- The file is served by github.com, the publisher’s own domain
- The download link uses HTTPS
- A SHA256 hash is published, so you can verify the file:
F2727924C77D44CC79431CEDF1B03A5649E0293F2E8F73D94C108B611ADFF0BD· Open the VirusTotal report for this hash - The link has not been checked yet
- The source code is public
- ExeAtlas does not host, wrap or modify installers
Related guide: How to check a file’s SHA256 hash on Windows
Recent versions
All 1 versions ›| Version | Date | Size | Files |
|---|---|---|---|
| 1.0.0Latest | September 29, 2026 | — | 64-bit MSI |
Install from the command line
Silent install ›winget install --id aleksandrreva-eng.llm-test-bench -emsiexec /i LLMTestBench-1.0.0.msi /qn /norestartmsiexec /x {C6400931-BC80-4D9D-9CC4-8EF5DD6D331E} /qn /norestartRelated guide: How to use winget: install and update apps
Information
- Publisher
- aleksandrreva-eng
- Category
- AI assistants & LLMs
- Licence
- MIT
- Price
- Free
- Requires
- Windows 11 / 10
- Architecture
- 64-bit
- Installer
- MSI
- Version
- 1.0.0
- Released
- September 29, 2026
- Added to the catalogue
- November 20, 2026
- Versions listed
- 1
- Downloads on ExeAtlas
- 1
- Website
- github.com
- winget ID
- aleksandrreva-eng.llm-test-bench
- MSI product code
- {C6400931-BC80-4D9D-9CC4-8EF5DD6D331E}
Ratings and reviews
No reviews yet. Be the first to rate it.
Alternatives to LLM Test Bench
See all ›-
Codex CLI0.171.0Codex CLI is an open‑source local coding agent that runs in your terminal, letting you write, edit, and understand code without leaving the command line.Open source20.2K ↓ -
Claude Code2.1.329Unleash Claude’s raw power directly in your terminal. Search million-line codebases instantly. Turn hours-long workflows into a single command. Your tools. Your workflow. Your codebase, evolving at thought speed.Proprietary73.6K ↓ -
Open source17.1K ↓ -
Open source8.7K ↓ -
Open source3.5K ↓ -
oh-my-pi19.0.1⌥ AI Coding agent for the terminal — hash-anchored edits, optimized tool harness, LSP, Python, browser, subagents, and moreOpen source1.8K ↓ -
Open source14.9K ↓ -
Open source950 ↓
Frequently asked questions
Is LLM Test Bench free?
Yes. LLM Test Bench is open-source software under the MIT licence.
Is LLM Test Bench safe to download?
The download comes from github.com, which belongs to the publisher; ExeAtlas does not host or change the file. Compare its SHA256 hash with the one shown here, or open the VirusTotal report for that hash.
How do I install LLM Test Bench with winget?
Open PowerShell or Windows Terminal and run the winget command shown on this page. winget downloads the official installer and checks its SHA256 hash before running it.
Can LLM Test Bench be installed silently?
Yes. The silent install page lists the exact command for each installer, including msiexec for the MSI package.
Does LLM Test Bench run on ARM (Windows on Arm)?
There is no ARM64 build; Windows 11 on Arm can run the 64-bit version through emulation.
Can I use LLM Test Bench without administrator rights?
Yes, there is a per-user installer that does not need administrator rights.
What are alternatives to LLM Test Bench?
Codex CLI, Claude Code, ChatGPT by lencx, GPT4All and Jan. The alternatives page compares them.