BOAT
Download

Models

Installing and managing models

Model references and tags, the install lifecycle, per-model devices, memory scheduling and where models are stored.

Models are independent packages. Core never imports model code; each installed model runs in its own isolated Python process, a worker, and the editor discovers new tools without a restart.

Model references

Models are referenced like container images:

Reference Meaning
sam3-litetext The model's default tag
sam3-litetext:S1 A specific tag

A tag is an installable variant, such as a size (S0, S1, L) or a quantization (3b-Q4_K_M, 3b-BF16). Each model page lists its tags, their download sizes and supported platforms. --tag S1 is the same as writing :S1.

Tags install side by side, so you can keep a small and a large variant of the same model. sam3-litetext and sam3-litetext:S0 refer to the same slot when S0 is the default, so installing either one twice does nothing.

Lifecycle

boat model install sam3-litetext         # install the default tag, enabled
boat model install sam3-litetext:L       # add another tag
boat model list                          # what is installed
boat model update sam3-litetext:L        # move to the latest compatible version
boat model disable sam3-litetext:L       # stop using it, keep the files
boat model enable sam3-litetext:L
boat model remove sam3-litetext:L        # delete code, environment and weights
Command What happens
install Downloads the latest compatible version, validates it with a real inference and enables it. If it is already installed, nothing changes and BOAT suggests update.
update Installs and validates the new version, then switches over and deletes the old one. Enabled/disabled state and the original install date are kept. On failure the previous version stays active.
disable Blocks new inference and stops the running worker. Files stay.
enable Checks the required files and environment, then makes the model available. Loading happens on first use.
remove Stops the worker and deletes the package code, environment, weights and install record, plus weight caches no other install uses.
list Works offline. Shows each model:tag, install date, state, version, effective device and idle time. Add --json for scripts.

When more than one tag of a model is installed, lifecycle commands need an explicit tag.

Downloads

Downloads are split into the code archive and the weights, each with a live progress bar showing file name, percentage, size, speed and time remaining. Before downloading, BOAT reports how much it can reuse from files it has already verified and how much it actually needs to fetch.

Weights are verified by size and SHA-256 and stored once in a content-addressed cache, then hard-linked into each install. Files that already exist are reused only if they match exactly. If local files exist but fail verification, the install stops and explains why instead of overwriting them.

License acceptance

Some models, such as SAM3, require you to accept their license before the first install. BOAT shows the terms and asks for confirmation. In scripts, accept up front:

boat model install sam3 --accept-license

Per-model device and idle time

By default every model uses the Default Device and unloads after 300 seconds of inactivity. Change either per installed tag:

boat model setup                         # interactive
boat model setup sam3-litetext:S0 --non-interactive --device cuda:1 --idle-seconds 600
  • --device accepts auto, cpu, mps, cuda:<n>, or default to inherit the Default Device.
  • --idle-seconds 0 unloads the worker after every request.

If a device change is interrupted, finish it with boat model setup recover.

A model only runs on the device you chose. If that device is unavailable, the request fails with a clear error instead of silently moving to another device.

Memory scheduling

Requests are serialized per device. Before starting a worker, BOAT reserves the model's inference weights plus 0.5 GB on the target device. If there is not enough room, it unloads idle workers from other models first, and if the space can never fit, it fails right away with an explicit error. Several tools from the same model share one resident worker.

The VRAM figure on each model page is the weight footprint across that model's tags, so budget about 0.5 GB on top.

Storage layout

Models live in a model-packages folder inside BOAT's data directory ($BOAT_HOME/data/model-packages when BOAT_HOME is set):

model-packages/
  .blobs/<sha256>                shared, verified weight files
  <model_id>/variants/<tag>/
    active.json                  version, tag, paths, install dates
    disabled                     present only when this tag is disabled
    versions/<version>-<digest>/
      package/                   shared files + this platform's runtime
      variant.json               resolved tag, profile, config and weights
      weights/                   this tag's weights (hard links)
      env/                       isolated Python environment

Local packages

To install a package from a folder, for example while developing one, pass a path:

boat model install ./model-packages/my-detector

Model IDs never take an @version. Publish a new version to update a package.

↑↓ navigate↵ open