LLM检查工具

Installation

```bash

Install globally

npm install -g llm-checker

1) Install

npm install -g llm-checker

Important: Use npm for Latest Builds

If you need the newest release, install from npm (llm-checker), not the scoped GitHub Packages mirror.

If you installed @pavelevich/llm-checker and version looks old:

npm uninstall -g @pavelevich/llm-checker
npm install -g llm-checker@latest
hash -r
llm-checker --version

Optional (Legacy): Install from GitHub Packages

Use this only if you explicitly need GitHub Packages. It may not match npm latest.

```bash

2) Install

npm install -g @pavelevich/llm-checker@latest ```

---

Setup (One Command)

```bash

Install globally first

npm install -g llm-checker

Print (or apply) the setup for your client

llm-checker mcp-setup --client claude # default; claude mcp add ... llm-checker mcp-setup --client codex # ~/.codex/config.toml llm-checker mcp-setup --client grok # ~/.grok/config.toml llm-checker mcp-setup --client kimi # ~/.kimi/mcp.json llm-checker mcp-setup --client cursor # ~/.cursor/mcp.json llm-checker mcp-setup --client windsurf # ~/.codeium/windsurf/mcp_config.json llm-checker mcp-setup --client gemini # ~/.gemini/settings.json llm-checker mcp-setup --client generic # raw mcpServers JSON for any client

`--apply` merges the server entry into the client's config file (existing
content is never clobbered), and `--json` prints the structured snippet for
scripting. `--npx` avoids a global server install by generating the explicit
package-qualified command below (the executable is part of `llm-checker`; there
is no separate `llm-checker-mcp` npm package):

Restart your client and you're done.

Verify every installed model's local blob while ranking them

llm-checker installed --verify llm-checker installed --verify --json

Quick Start

```bash

Calibration Quick Start (10 Minutes)

This path produces both calibration artifacts and verifies calibrated routing in one pass.

Example Prompts

After setup, you can ask your assistant things like:

• "What's the best coding model for my hardware?"
• "Benchmark qwen2.5-coder and show me the tok/s"
• "Compare llama3.2 vs codellama for coding tasks"
• "Clean up my Ollama — what should I remove?"
• "What model should I use for this Rust project?"
• "Optimize my Ollama config for maximum performance"
• "How much RAM is Ollama using right now?"

• "Verify the blob of the model I just downloaded before running it"

Your assistant will automatically call the right tools and give you actionable results.

---

Integration Examples (SIEM / CI Artifacts)

```bash

Scoring Weights by Use Case

Three scoring systems are available, each optimized for different workflows:

Deterministic Selector (primary — used by check and recommend):

Scoring Engine (used by search for catalog scoring; smart-recommend's final ranking is produced by the shared scoring core — see #96):

All weights are centralized in src/models/scoring-config.js.

Examples

Detect your hardware:

llm-checker hw-detect

Get recommendations for all categories:

llm-checker recommend

Full system analysis with compatible models:

llm-checker check

Find the best coding model:

llm-checker recommend --category coding

Search for small, fast models under 5GB:

llm-checker search "7b" --max-size 5 --use-case fast

Get high-quality reasoning models:

llm-checker smart-recommend --use-case reasoning

---

1) Use the sample prompt suite

cp ./docs/fixtures/calibration/sample-suite.jsonl ./sample-suite.jsonl

1) Configure registry + token (PAT with read:packages)

echo "@pavelevich:registry=https://npm.pkg.github.com" >> ~/.npmrc echo "//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}" >> ~/.npmrc

Export all configured formats (json, csv, sarif)

llm-checker audit export --policy ./policy.yaml --command check --format all --out-dir ./reports ```

• --command check|recommend chooses the candidate source.
• --format all honors reporting.formats in your policy (falls back to json,csv,sarif).
• In enforce mode with blocking violations, reports are still written before non-zero exit.

4) Refresh the catalog when you want current Ollama references

llm-checker sync

Interactive CLI Panel

Running llm-checker with no arguments now opens an interactive panel (TTY terminals):

• animated startup banner
• main command list with descriptions
• type / to open all commands
• use up/down arrows to select a command
• press Enter to execute
• add optional extra flags before run (example: --json --limit 5)

For scripting and automation, direct command invocation remains unchanged:

llm-checker check --use-case coding --limit 3
llm-checker search "qwen coder" --json

---

CI artifact (JSON) for post-processing in pipeline jobs

llm-checker audit export --policy ./policy.yaml --command check --format json --out ./reports/policy-report.json

Flat CSV for SIEM ingestion (Splunk/ELK/DataDog pipelines)

llm-checker audit export --policy ./policy.yaml --command check --format csv --out ./reports/policy-report.csv

SARIF for security/code-scanning tooling integrations

llm-checker audit export --policy ./policy.yaml --command check --format sarif --out ./reports/policy-report.sarif ```

Component Responsibilities

• Input layer: Collects runtime constraints from hardware detection, local inventory, dynamic registry data, and CLI flags.
• Normalization layer: Deduplicates identifiers/tags and builds a canonical candidate set.
• Selection layer: Filters by use case, selects the best fitting quantization, and computes deterministic Q/S/F/C scores.
• Governance layer: Applies policy rules in audit or enforce mode and records explicit violation metadata.
• Output layer: Returns ranked recommendations plus machine-readable compliance artifacts when requested.

Comparison with Other Tooling (e.g. `llmfit`)

LLM Checker and llmfit solve related but different problems:

If your goal is: "What should I run on this exact machine right now?", use LLM Checker first. If your goal is broader experimentation across custom pipelines, using both tools can be complementary.

---