Dragon CLI

Run open-weight LLMs on your own machine, give them tools through MCP, and plug them into any agent framework that speaks the OpenAI API.

pip install dragon-cli
dragon-cli-install-llm   # picks the right llama.cpp build for your CPU/GPU
dragon-cli

Get started Use with agents GitHub

What it does

One command line tool that covers the whole local-LLM loop: choosing a model that fits your hardware, downloading it, chatting with it, giving it tools, and serving it to other apps.

Local streaming chat

Talk to GGUF models through llama.cpp with tokens streamed as they're generated. No cloud API key, nothing leaves your machine.

CPU or GPU, auto-tuned

Detects NVIDIA (CUDA), AMD (ROCm) and Apple Silicon (Metal). Offloads as many layers as your VRAM allows and backs off gracefully when it runs out.

Hardware-aware recommendations

Profiles your RAM, CPU and GPU, then ranks models by speed, quality, balanced or memory. --auto picks for you.

Hugging Face hub

A curated catalog (Qwen 2.5, Llama 3.x, DeepSeek-R1 distills and more), plus search across the Hub. Add any GGUF by repo id, URL or hf:// path.

MCP tool-calling

Connect Model Context Protocol servers (filesystem, git, fetch, SQLite, memory, time, or your own) and the model calls their tools mid-conversation.

Conversation history

Every message is saved to SQLite the moment it's sent. Resume, rename, search, delete, or export a conversation to Markdown or JSON.

OpenAI-compatible server

--serve exposes /v1/chat/completions (with streaming) and /v1/models, plus Swagger docs and a browser chat UI.

Interactive menu

Run dragon-cli with no flags for a full-screen menu: arrow keys, number shortcuts, and slash commands like /hub or /serve.

Install with pip

Requires Python 3.10 or newer, on Windows, Linux or macOS. Using a virtual environment is recommended.

Create a virtual environment (optional but recommended)

python -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate

Install Dragon CLI from PyPI

pip install dragon-cli

To upgrade later: pip install -U dragon-cli.

Install the inference engine

llama-cpp-python isn't a hard dependency because on many platforms PyPI only has its C++ source, so a plain install would try to compile it. This helper detects your OS and GPU and installs a prebuilt wheel (CPU, CUDA, ROCm or Metal) instead:

dragon-cli-install-llm

Already have a C/C++ toolchain? pip install "dragon-cli[llm]" builds it from source instead.

Run it

dragon-cli                 # interactive menu
dragon-cli --auto          # pick the best model for this machine and start chatting

Models, mcp_servers.json and dragon_cli.db are created in the folder you run it from.

Other ways to install

# latest development version straight from GitHub
pip install git+https://github.com/JPEDROPS092/dragon-cli-ai-local

# from a local clone, editable
git clone https://github.com/JPEDROPS092/dragon-cli-ai-local
cd dragon-cli-ai-local
pip install -e .

Usage

Use the interactive menu, or pass a flag to go straight to a specific action.

Chat and models

dragon-cli --list                # show the model catalog
dragon-cli --model 5             # chat with catalog model #5
dragon-cli --auto --priority speed
dragon-cli --resume 3            # continue conversation #3
dragon-cli --no-mcp              # chat without tools

Hardware

dragon-cli --profile             # CPU, RAM, GPU, VRAM
dragon-cli --recommend --priority quality
dragon-cli --gpu                 # offload all layers
dragon-cli --cpu                 # ignore the GPU
dragon-cli --ngl 20              # offload exactly 20 layers

HTTP server

dragon-cli --serve --model 5
# → http://127.0.0.1:8000/v1     OpenAI API
# → http://127.0.0.1:8000/chat   browser chat
# → http://127.0.0.1:8000/docs   Swagger UI

dragon-cli --serve --host 0.0.0.0 --port 8080

History

dragon-cli --list-conversations
dragon-cli --db ~/notes/ai.db    # separate database

# inside a chat:
/search docker
/export 3 md

Guides: what you'll see in the terminal

Step-by-step walkthroughs for each feature, with the output Dragon CLI prints. Paths, hardware and model names will differ on your machine.

1. Install the inference engine dragon-cli-install-llm

Run this once after pip install dragon-cli. It detects your OS, Python version and GPU, and if a GPU is found it asks which backend to use (cpu, cuda, rocm or mps). Press Enter to keep CPU.

terminal

$ dragon-cli-install-llm
[dragon-cli] Detected Linux (x86_64), Python 3.12.7

[dragon-cli] Detected NVIDIA GPU via nvidia-smi: NVIDIA GeForce RTX 3060
[dragon-cli] GPU detected. Do you want to use a GPU-enabled wheel?
[dragon-cli] Default backend: CPU
[dragon-cli] Available backends: cpu, cuda, rocm, mps
[dragon-cli] Choose backend [cpu]: cuda
[dragon-cli] Installing llama-cpp-python from prebuilt CUDA wheels (...)...

[dragon-cli] ✓ llama-cpp-python installed successfully!
[dragon-cli] GPU backend selected (CUDA)

If no prebuilt wheel exists for your platform, it prints ✗ Installation failed. followed by the compiler to install for your OS and the fallback command.

2. The interactive menu dragon-cli

With no flags, Dragon opens a full-screen menu showing your hardware and the model it recommends. Move with the arrow keys and press Enter, press 1–8 as shortcuts, / to type a command (/chat, /hub, /serve, /mcp, /history, /profile), r to refresh, or q to quit.

dragon-cli

╭──────────────────────────────────────────────────────────╮
│             DRAGON CLI  v0.2.1   100% local              │
╰──────────────────────────────────────────────────────────╯
╭─ System status ───────────────────────────────────────────╮
│  System       Fedora Linux 44 x86_64                      │
│  CPU          Intel Core i7-1185G7  (8 cores)             │
│  RAM          13.8 GB free / 31.1 GB total                │
│  GPU          not detected (CPU mode)                     │
│  Backend      CPU                                         │
│  Recommended  Qwen2.5-1.5B-Instruct (Q4_K_M)  (100%)      │
╰──────────────────────────────────────────────────────────╯
╭─ Main menu ──────────────────────────────────────────────╮
│  1 › Chat with model      Talk to a local model         │
│  2   List models          Browse the catalog            │
│  3   Auto recommendation  Best model for your hardware  │
│  4   Connect to hub       Search Hugging Face           │
│  5   Manage history       List, resume, export          │
│  6   Serve HTTP           OpenAI-compatible API         │
│  7   Manage MCP servers   Add, remove, test             │
│  8   Quit                 Exit Dragon CLI               │
╰──────────────────────────────────────────────────────────╯
   Arrows move  Enter select  1-8 shortcut  / commands  q quit

3. Your first chat dragon-cli --auto

--auto picks the best model for your hardware (or use --model N to choose one).
The model is downloaded from Hugging Face into ./models. This happens only once; the next run says Model already saved.
It's loaded with thread count, context and GPU layers tuned for your machine.
Type a message after You:. The answer streams in token by token, and every message is saved to dragon_cli.db.

terminal

$ dragon-cli --auto

Auto-selected model based on your hardware:
  → Qwen2.5-1.5B-Instruct (Q4_K_M) — tiny, GPU-friendly
  Why: ✓ Plenty of RAM (13.8 GB available, 3 GB needed); ○ CPU-only mode

[download] Downloading qwen2.5-1.5b-instruct-q4_k_m.gguf (a few GB, only happens once)...
[ok] Saved to models/qwen2.5-1.5b-instruct-q4_k_m.gguf
[load] Loading model (threads=8, ctx=8192, mode=CPU)...
[mcp] No servers configured in mcp_servers.json.

=== Chat started — Qwen2.5-1.5B-Instruct (Q4_K_M) on CPU | MCP off — /help for commands, /exit to quit ===

You: what is a GGUF file?
GGUF is a binary file format used by llama.cpp to store a model's weights,
tokenizer and metadata in a single file, usually quantized...

You: /exit
Shutting down. See you next time!

4. Choose a model that fits your hardware --profile · --recommend · --list

--profile shows what Dragon detected. --recommend scores every catalog model against it; add --priority speed|quality|balanced|memory to change the ranking. --list prints the numbered catalog, which you pass to --model N.

terminal

$ dragon-cli --profile
============================================================
HARDWARE PROFILE
============================================================
CPU:
  Model: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
  Cores: 8 logical
Memory (RAM):
  Total: 31.06 GB
  Available: 13.82 GB
GPU:
  No dedicated GPU detected (CPU-only mode)
============================================================
EXECUTION CAPACITY ASSESSMENT
============================================================
✓ Memory Adequate: 13.8 GB available
  → Can run models up to 14B+ (quantized)

$ dragon-cli --recommend --priority balanced
Top Recommendations:
  1. Qwen2.5-1.5B-Instruct (Q4_K_M) — tiny, GPU-friendly
     [RECOMMENDED]
     Match: ██████████ 100%
     Min RAM: 3 GB | Context: 8192
  ...

$ dragon-cli --list
  5. Qwen2.5-7B-Instruct (Q4_K_M) — general use, great for tools  [cloud] (GPU full ~5GB)
  6. DeepSeek-R1-Distill-Qwen-7B (Q4_K_M) — reasoning/code  [cloud] (GPU full ~5GB)
  8. Qwen2.5-14B-Instruct (Q4_K_M) — heavy, best quality  [cloud] [!] RAM may be too low right now

[cloud] means not downloaded yet; models already on disk show [downloaded].

5. Add any model from Hugging Face menu → 4 Connect to hub

Open dragon-cli and pick Connect to hub (or type /hub).
Type a catalog number to download it, s to search the Hub, or paste an hf:// path.
The search accepts a free-text query, an owner/repo id or a full Hugging Face URL. Dragon lists the GGUF files, pre-selects a sensible quantization, estimates the RAM needed, and adds the model to your catalog.

dragon-cli → Connect to hub

Hub action (catalog # download, 's' search/add, 'hf://' add direct, Enter = back): s
Search Hugging Face (query, owner/repo, hf://, or HF URL): phi-3.5 mini gguf
  ...
Added as catalog #9: Phi-3.5-mini-instruct (Q4_K_M)
Download this model now? [y/n] (y): y
[download] Downloading Phi-3.5-mini-instruct-Q4_K_M.gguf (a few GB, only happens once)...
[ok] Saved to models/Phi-3.5-mini-instruct-Q4_K_M.gguf

Custom models are stored in the catalog, so they show up in --list and can be used with --model 9.

6. Let the model use tools (MCP) mcp_servers.json · /tools

Add servers from the menu (Manage MCP servers → add) or edit mcp_servers.json. See MCP tools below.
Start a chat. Each server is launched and its tools are announced to the model.
Ask something that needs a tool. The model emits a tool call, Dragon runs it, shows a preview of the result, and the model answers using it. Up to five tool calls can chain in one turn.

terminal

$ dragon-cli --model 5
[load] Loading model (threads=8, ctx=8192, mode=CPU)...
[mcp] 'filesystem' connected (11 tools)
[mcp] 'time' connected (2 tools)

=== Chat started — Qwen2.5-7B-Instruct (Q4_K_M) on CPU | 13 MCP tools — /help for commands, /exit to quit ===

You: which python files are in this folder?
[tool] running filesystem.list_directory({'path': '.'})
[tool] result: [FILE] dragon_cli.py\n[FILE] pyproject.toml\n[DIR] src ...
There is one Python file at the top level, dragon_cli.py. The package code
lives in the src/ folder.

A server that fails to start prints [mcp] Failed to connect 'name': ... and the chat continues without it. Use /mcp to see what's connected and /tools to list tool names.

7. Find, resume and export conversations /list · /search · /resume · /export

Conversations live in dragon_cli.db in the current folder. Manage them inside a chat with slash commands, from the menu (Manage history), or with --list-conversations and --resume ID.

chat

You: /list
  #3 *  GGUF questions  (6 msgs, updated 2026-10-22 10:14:03)
  #2  Docker compose help  (14 msgs, updated 2026-10-21 18:40:51)
  #1  First chat  (2 msgs, updated 2026-10-20 09:02:17)

You: /search compose
  #2 Docker compose help [user]: how do I add a healthcheck to my compose file...

You: /rename 2 Docker healthchecks
[ok] Renamed #2 to 'Docker healthchecks'.

You: /export 2 md
[ok] Exported to /home/you/project/conversation_2.md

You: /resume 2
[ok] Resumed conversation #2.

* marks the active conversation. /delete asks for confirmation before removing anything.

8. Serve the model to other apps dragon-cli --serve

Instead of chatting, load a model and expose it over HTTP. Open the printed /chat URL for a browser chat, or connect an agent framework. Stop with Ctrl+C.

terminal

$ dragon-cli --serve --model 5 --port 8000
[ok] Model already saved at models/qwen2.5-7b-instruct-q4_k_m.gguf
[load] Loading model (threads=8, ctx=8192, mode=CPU)...
[serve] OpenAI-compatible endpoint at http://127.0.0.1:8000/v1
[serve] Interactive docs (Swagger UI): http://127.0.0.1:8000/docs
[serve] Chat UI: http://127.0.0.1:8000/chat
[serve] Model: Qwen2.5-7B-Instruct (Q4_K_M) on CPU. Press Ctrl+C to stop.

Requests are processed one at a time (llama.cpp isn't thread-safe), so concurrent calls wait in line.

9. Tune GPU offload --gpu · --cpu · --ngl

By default Dragon decides how many layers fit in your VRAM. If loading runs out of memory, it retries with fewer GPU layers and finally falls back to CPU instead of crashing.

terminal

$ dragon-cli --model 8 --gpu
[load] Loading model (threads=8, ctx=8192, mode=GPU)...
[gpu] Offloading 48 layers failed (likely out of VRAM). Retrying with fewer layers...
[gpu] Running with 32/48 layers on GPU (partial offload).

$ dragon-cli --model 8 --ngl 20    # exactly 20 layers on the GPU
$ dragon-cli --model 8 --cpu       # ignore the GPU entirely

If you installed the CPU wheel, every GPU option ends with [gpu] GPU offload unavailable; running on CPU. Run dragon-cli-install-llm again and pick a GPU backend.

Use it with agent frameworks

Start the server with dragon-cli --serve, then point any client that speaks the OpenAI API at http://127.0.0.1:8000/v1. The API key can be any non-empty string, and the model name is ignored because the server hosts a single model.

pip install openai

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="dragon")

stream = client.chat.completions.create(
    model="local",
    messages=[{"role": "user", "content": "Explain GGUF in one paragraph."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

pip install openai-agents. Tracing is turned off so the SDK doesn't look for an OpenAI key.

from openai import AsyncOpenAI
from agents import Agent, OpenAIChatCompletionsModel, Runner, set_tracing_disabled

set_tracing_disabled(True)
client = AsyncOpenAI(base_url="http://127.0.0.1:8000/v1", api_key="dragon")

agent = Agent(
    name="Assistant",
    instructions="You are a concise technical assistant.",
    model=OpenAIChatCompletionsModel(model="local", openai_client=client),
)

result = Runner.run_sync(agent, "Write a haiku about local inference.")
print(result.final_output)

pip install langchain-openai

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://127.0.0.1:8000/v1",
    api_key="dragon",
    model="local",
    temperature=0.7,
)
print(llm.invoke("Give me three names for a dragon.").content)

pip install llama-index-llms-openai-like

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="local",
    api_base="http://127.0.0.1:8000/v1",
    api_key="dragon",
    is_chat_model=True,
)
print(llm.complete("What is retrieval-augmented generation?"))

pip install crewai. The openai/ prefix tells CrewAI to use the OpenAI-compatible route.

from crewai import Agent, Crew, LLM, Task

llm = LLM(model="openai/local", base_url="http://127.0.0.1:8000/v1", api_key="dragon")

writer = Agent(role="Writer", goal="Write clear docs", backstory="A technical writer.", llm=llm)
task = Task(description="Summarize what Dragon CLI does.", expected_output="One paragraph.", agent=writer)

print(Crew(agents=[writer], tasks=[task]).kickoff())

npm install openai

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://127.0.0.1:8000/v1", apiKey: "dragon" });

const res = await client.chat.completions.create({
  model: "local",
  messages: [{ role: "user", content: "Hello from Node!" }],
});
console.log(res.choices[0].message.content);

The Continue coding assistant for VS Code and JetBrains. Add this to ~/.continue/config.yaml:

models:
  - name: Dragon (local)
    provider: openai
    model: local
    apiBase: http://127.0.0.1:8000/v1
    apiKey: dragon

The Aider pair-programming CLI:

aider --model openai/local \
      --openai-api-base http://127.0.0.1:8000/v1 \
      --openai-api-key dragon

In Open WebUI, go to Admin Settings → Connections → OpenAI API and add a connection:

URL:     http://host.docker.internal:8000/v1    # or http://127.0.0.1:8000/v1 without Docker
API key: dragon

If Open WebUI runs in Docker, start Dragon with --host 0.0.0.0 so the container can reach it.

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}], "stream": true}'

What the server supports. It implements messages, temperature, max_tokens and stream. It does not implement OpenAI function calling (the tools parameter), so agents that rely on calling your Python functions will chat normally but won't run those functions. If you need tools, use MCP inside Dragon's own chat (below). The server binds to 127.0.0.1 by default and has no authentication, so only use --host 0.0.0.0 on networks you trust.

Give the model tools with MCP

In the interactive chat, Dragon connects to the MCP servers listed in mcp_servers.json and teaches the model to call their tools. The result of each call goes back to the model before it answers.

Add servers from the menu

Open dragon-cli → Manage MCP servers (or type /mcp), then add, remove or test. Built-in presets:

Preset	What it gives the model
`filesystem`	Read and write files in a folder you choose
`fetch`	Fetch and read web pages
`git`	Inspect and operate on a local repository
`memory`	A persistent knowledge-graph memory
`sqlite`	Query a local SQLite database
`time`	Current time and timezone conversion

Or edit the config by hand

Same format used by Claude Desktop and other MCP clients, so any stdio server works:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "."]
    },
    "git": {
      "command": "uvx",
      "args": ["mcp-server-git", "--repository", "."]
    }
  }
}

npx needs Node.js and uvx needs uv. In a chat, /mcp shows connection status and /tools lists every tool. For tool use, 7B+ instruct models (Qwen 2.5 7B is a good start) work much better than tiny ones.

Reference

In-chat commands

Command	Description
`/help`	Show available commands
`/models`	List the model catalog
`/tools`	List connected MCP tools
`/mcp`	Show MCP server connection status
`/clear`	Clear the current in-memory history
`/new [title]`	Start a new conversation
`/list`	List saved conversations
`/resume <id>`	Resume a saved conversation
`/rename <id> <title>`	Rename a conversation
`/delete <id>`	Delete a conversation
`/search <text>`	Search saved messages
`/export <id> [md\|json]`	Export a conversation to a file
`/exit`, `/quit`	Quit

HTTP endpoints (`--serve`)

Endpoint	Description
`POST /v1/chat/completions`	OpenAI-style chat completion, streaming (SSE) or not
`GET /v1/models`	The loaded model
`GET /chat`	Browser chat UI
`GET /docs`, `/redoc`	Interactive API docs
`GET /openapi.json`	OpenAPI schema

All CLI flags

Flag	Description
`--list`	List the model catalog and exit
`--model N`	Use catalog model number N
`--auto`	Pick the best model for your hardware
`--priority`	`speed`, `quality`, `balanced` (default) or `memory`
`--recommend`	Show recommendations and exit
`--profile`	Show the hardware profile and exit
`--gpu` / `--cpu`	Force full GPU offload, or CPU only
`--n-gpu-layers N`, `--ngl N`	Layers to offload (-1 auto, 0 CPU)
`--no-mcp`	Don't connect to MCP servers
`--resume ID`	Resume a saved conversation
`--list-conversations`	List saved conversations and exit
`--db PATH`	Use a different SQLite database
`--serve`	Serve the model over HTTP instead of chatting
`--host`, `--port`	Bind address (default `127.0.0.1`) and port (default `8000`)

What it does

Local streaming chat

CPU or GPU, auto-tuned

Hardware-aware recommendations

Hugging Face hub

MCP tool-calling

Conversation history

OpenAI-compatible server

Interactive menu

Install with pip

Create a virtual environment (optional but recommended)

Install Dragon CLI from PyPI

Install the inference engine

Run it

Other ways to install

Usage

Chat and models

Hardware

HTTP server

History

Guides: what you'll see in the terminal

Use it with agent frameworks

Give the model tools with MCP

Add servers from the menu

Or edit the config by hand

Reference

In-chat commands

HTTP endpoints (--serve)

All CLI flags

HTTP endpoints (`--serve`)