Junie Local

Prerequisites

The requirements depend on your platform:

Inference engines

Junie Local uses a different inference engine on each platform, so the model files and the installed engine differ:

• macOS: mlx-vlm, an open-source framework for running vision-language models on Apple Silicon. Junie Local packages it as a managed background server (junie-mlx-vlm) and installs the MLX builds of the models.

• Windows: llama.cpp with CUDA acceleration. Junie Local installs the GGUF builds of the same models.

Both engines use multi-token prediction (MTP) for speculative decoding, which can speed up generation. On macOS, it comes from a small separate draft model; on Windows, the main GGUF model provides it.

Supported models

Two models are available:

• Qwen3.6-27B: the smaller and faster of the two, usable on a laptop with reasoning disabled.

• Qwen3.8-3.6-27B-blend: a JetBrains model that merges the weights of Qwen3.8-27B and Qwen3.6-27B in equal proportions. It completes more coding tasks than Qwen3.6 while generating far fewer output tokens than Qwen3.8. See Making Local AI Smarter and Faster for the benchmarks.

The links above point to the original full-weight models. Junie Local does not download those: it installs 4-bit quantized builds of them, so the models fit into the memory of a laptop or a single GPU. The quantized build is chosen for your engine — MLX 4-bit on macOS, Q4_K_M GGUF on Windows.

Quick start

If Junie CLI is already installed, run the /local command in an active session and pick a model on the setup screen.

To install Junie CLI together with the local model in one run, use the Junie installer with --local-model:

The installer downloads the inference engine and the model (~15 GB on macOS, ~17 GB on Windows), verifies the checksums, and writes the Junie CLI model configuration. Then start Junie CLI, run /model, and pick the installed local model. The first response waits for the model to finish loading; later ones are faster.

Install Junie Local

The /local command opens the setup screen inside Junie: it checks the system requirements first, then asks the installer which models are available and lets you pick one. Models that are already installed are marked as such and selecting one reinstalls it. The installer then runs and reports its progress on the same screen; keep the screen open until the installation is done.

When at least one model is installed, /local opens the list of installed models instead. Choose Install another model there to go through the flow again and add one more; the models live side by side.

In ACP clients on macOS, /local has no screen of its own to draw: it opens a Terminal window and runs the installer there. That installation keeps going even if the client exits, and the window closes itself once the script is done. On Windows, /local is available only in the Junie CLI terminal UI.

Downloads are resumable and every unpacked artifact is marked as installed, so a re-run only fetches what is missing.

Each model version installs separately and gets its own Junie CLI model configuration, so the versions live side by side and installing one leaves the other untouched.

After installation, the models are available in the models directory of the Junie Local installation. The layout depends on the engine:

• macOS, ~/.local/share/junie-local/models/, one directory per model:

  • Qwen3.6-27B-MLX-4bit/: the main model (~15 GB).

  • Qwen3.6-27B-MTP-MLX-4bit/: the draft model for speculative decoding (~250 MB).

• Windows, ~\.local\share\junie-local\models\, one GGUF file per artifact:

  • Qwen3.6-27B-Q4_K_M.gguf: the main model (~16 GB).

  • mmproj-Qwen3.6-27B-Q8_0.gguf: the multimodal projector (~0.6 GB).

Junie CLI is configured automatically: the installer writes a custom model profile to $JUNIE_HOME/models/ that points at http://localhost:19239/v1/chat/completions. The /local setup screen selects the new model in the running session; after a standalone installer run, run /model in Junie CLI and pick the installed model. You can switch models later with the same command.

Engine control

Junie starts the engine itself whenever the local model is selected, including after a reboot, and stops it once the last Junie instance using it exits. The engine keeps loading the model in the background after it starts, so the first request through Junie waits for it to become ready.

The bundled serverctl script is only needed to control the engine outside Junie. On macOS it is ~/.local/share/junie-local/current/serverctl.sh; on Windows it is serverctl.ps1 in the installed engine version directory under ~\.local\share\junie-local\versions\:

To remove one model, open its entry on the /local screen and choose Remove. Junie runs serverctl uninstall <model> and deletes the generated model configuration, leaving the engine and the other installed models in place. When the removed model is the last one the engine serves, Junie falls back to serverctl uninstallAll instead, so no engine is left running without a model.

To remove Junie Local entirely, choose Remove all models on the /local screen. Junie runs serverctl uninstallAll, which stops the engine and deletes the installation together with every generated model configuration.