Token catalog: how to author your own

The token catalog is a backend-driven UI feed: each provider lives in a JSON file, and devboy secrets ui renders the provision form by reading those files at runtime. Adding a new provider — or a new variant of an existing one (CN tier vs global tier vs subscription tier) — is a JSON edit, not a Rust patch.

This guide is for catalog authors: anyone shipping a provider entry, either upstream as a contribution to devboy-tools or downstream in their own team's repo.

Schema reference: crates/devboy-token-catalog/schema/v1.json (JSON Schema 2020-12, point your IDE at it for autocomplete + inline validation).

Why this exists

Most providers ship a single API key kind, and a single regex + retrieval URL is enough. Some don't:

Kimi has CN (api.moonshot.cn), Global (api.moonshot.ai), and Coding (api.kimi.com/coding) tiers — separate consoles, separate billing, separate regex prefixes for the Coding tier.
AWS has access-key pairs vs IAM role assumption vs SSO short-lived creds.
Hashicorp Cloud has user tokens vs service-principal tokens vs Vault root tokens.
Internal corporate providers often have prod / staging / sandbox endpoints with different procedures.

A single pattern_id per path can't capture this. The catalog formalises per-provider variants so the user picks the exact kind, sees the exact procedure for it, and the framework probes the exact endpoint.

Discovery sources

devboy-tools walks three sources in order, least-to-most specific:

Source	Path	Use
Bundled	compiled into the binary	Reference defaults shipped upstream (currently: Kimi).
User	`~/.devboy/secrets/catalog/*.json`	Per-machine entries the user maintains for themselves.
Project	`<project>/.devboy/secrets/catalog/*.json`	Team-shared entries, versioned with the project's manifest.

Later sources override earlier ones on provider_id collision. So a project-scope kimi.json wins over the user one, which wins over the bundled default. This lets a team pin its own canonical Kimi procedure (e.g. with internal-tooling extras) without forking devboy-tools.

File format

One file = one provider. The schema:

{
  "$schema": "https://devboy-tools.dev/schemas/token-catalog/v1.json",
  "schema_version": 1,
  "provider_id": "<kebab-case-id>",
  "display_name": "Human-readable name",
  "description": "One-paragraph context shown above the variant list.",
  "variants": [
    {
      "id": "<provider>-<variant>",
      "display_name": "Variant name",
      "description": "What this variant is and when to pick it.",
      "format_regex": "^...$",
      "format_hint": "human-readable shape hint",
      "retrieval": {
        "console_url": "https://console.example.invalid/keys",
        "docs_url": "https://docs.example.invalid/authentication",
        "steps": [
          "Step one — clear, action-oriented.",
          "Step two."
        ],
        "notes": "Optional gotchas, scope requirements, billing caveats."
      },
      "liveness": {
        "kind": "http",
        "url": "https://api.example.invalid/v1/probe",
        "method": "GET",
        "auth": { "kind": "bearer" },
        "expect_status": 200
      },
      "rotation": {
        "method": "manual",
        "every_days": 90,
        "guide_url": "https://docs.example.invalid/key-rotation",
        "notes": "How to rotate: create the replacement, deploy it, revoke the old. Note any overlap window or atomic-pair constraint."
      }
    }
  ]
}

Required fields

schema_version — pinned to 1.
provider_id — lowercase kebab-case, matches the filename without extension.
display_name — non-empty.
variants — at least one entry; each variant needs id, display_name, description, retrieval.
retrieval.console_url + retrieval.steps (≥ 1 step).

Optional fields

description (provider-level)
format_regex (Rust regex syntax, anchored)
format_hint (the human-readable counterpart)
liveness (HTTP probe — currently the only kind)
rotation (cadence + method)
default_keychain_account (overrides the default account = path convention)
retrieval.notes
retrieval.docs_url — the provider's official documentation for the credential (scopes, security best practices, auth model). Distinct from console_url: that's where you make the key, this is where you understand it. The provision dialog renders it as a "Provider docs" link.
rotation.guide_url — the provider's rotation / key-hygiene guide, when one exists. Rendered as a "Rotation guide" link.
rotation.notes — the concrete rotation procedure (the how, not the when): atomic pair rotation, reinstall-after-scope-change, overlap window, hard-cutover. Rendered as a block in the dialog's rotation section.

Bundled-catalog bar: the catalog crate's test suite enforces that every bundled variant carries retrieval.docs_url and either rotation.guide_url or a non-empty rotation.notes. A bare {method, every_days} rotation block tells the user when to rotate but not how — that's a regression the every_bundled_variant_has_rotation_guidance test fails on. User-authored and URL-loaded catalogs are not held to this bar (the fields stay optional in the schema), but following it is strongly recommended.

Authoring a new provider

Pick a provider_id that doesn't collide with bundled defaults. Run devboy secrets catalog list (when the command lands) to see what's already loaded.
Drop <provider>.json into one of the three discovery dirs. For team-shared entries, choose <repo>/.devboy/secrets/catalog/.

Add the $schema reference at the top so your editor validates while you type:

{ "$schema": "https://devboy-tools.dev/schemas/token-catalog/v1.json", ... }

Cover at least one variant. Most providers are single-variant; multi-variant is for those that genuinely have separate token kinds (region / tier / subscription).
Test with devboy secrets catalog validate path/to/file.json (when the command lands) — runs the same JSON-Schema validation the runtime does, plus URL liveness on console_url and liveness.url.

Authoring a new variant for an existing provider

Pick the right discovery scope:

Upstream PR — when the variant is a public, stable provider feature (e.g. Kimi adding a fourth subscription tier). Land it in data/<provider>.json of crates/devboy-token-catalog/.
User-scope — when it's specific to your account (e.g. a beta program key your team got under NDA). Drop into ~/.devboy/secrets/catalog/<provider>.json.
Project-scope — when the team needs a custom shape (corporate-prefixed regex, internal proxy URL). Drop into <repo>/.devboy/secrets/catalog/<provider>.json.

Project-scope is the one most teams will touch. It's safe to commit alongside the manifest because the file contains no secret values — only metadata about how to obtain them.

Worked example

crates/devboy-token-catalog/data/kimi.json ships three variants of the Moonshot AI provider (CN, Global, Coding). Each declares its own:

format_regex — CN/Global share ^sk-[A-Za-z0-9]{32,}$; Coding uses ^kc-[A-Za-z0-9]{40,}$.
retrieval.console_url — platform.moonshot.cn vs platform.moonshot.ai vs kimi.com/dev.
liveness.url — api.moonshot.cn vs api.moonshot.ai vs api.kimi.com/coding.
retrieval.notes — gotcha about not crossing keys between hosts (CN key against Global host = 401).

Read the JSON file end-to-end — it's the canonical reference for what a polished provider entry looks like.

What the GUI does with it

Reads the catalog at startup (bundled + user + project, in that order).
When a user opens the provision dialog for a path whose pattern_id matches a variant id, the dialog uses the catalog's format_regex for live feedback and liveness for the on-Save HTTP probe.
The context card above the input renders display_name, description, retrieval.console_url as a hyperlink, and the retrieval.steps as a numbered list — straight out of the JSON, no per-provider Rust code.
When the catalog file at the project scope overrides the bundled one, the GUI shows a small chip indicating which source the variant came from. That keeps it auditable when a team pins its own version.

Stability promise

schema_version: 1 is stable. Future major bumps land as v2.json alongside, and the loader keeps reading both for at least one minor release before old files become an error. Authors can pin their files to a known schema by URL and get notified when they need to migrate.

Once you have a working catalog file, the typical next step is sharing it with teammates so they don't have to re-author the same metadata. There are three layered options:

Disk-share — drop the JSON file into ~/.devboy/secrets/catalog/ on each developer machine (Ansible, dotfiles, make install). Simplest path; downside is keeping copies in sync after every edit.
Project-scope override — commit <project>/.devboy/secrets/catalog/<provider>.json to the project repo. Anyone who clones inherits the catalog automatically; the project-scope copy wins over user / bundled per the override precedence in token-catalog.md §discovery-sources.
URL source — host the JSON on a Git provider's raw endpoint (or any HTTPS server) and let devboy secrets catalog add-url subscribe each developer machine. The [[source]] entry lands in ~/.devboy/secrets/catalog/sources.toml; the loader fetches with sha-pin or TOFU, caches, and refreshes at the configured TTL. Right when more than one team uses the same provider knowledge.

Recommended layout for a community / team catalog repo

If you're standing up a shared repo (think devboy-catalog, <team>-secrets-catalog, …), keep the layout simple so any consumer can add-url against any file without touching the repo:

your-org/devboy-catalog/
├── README.md          ← short pitch + the canonical raw URL pattern
├── LICENSE
├── anthropic.json
├── openai.json
├── gitlab.json
├── slack.json
├── …                  ← one file per provider, mirrors crates/devboy-token-catalog/data/
└── .github/workflows/
    └── validate.yml   ← runs `devboy secrets catalog validate <file>` on every PR

The README.md should give the consumer the exact subscribe command:

devboy secrets catalog add-url \
  https://raw.githubusercontent.com/your-org/devboy-catalog/main/anthropic.json \
  --pin <sha256>            # recommended for prod; or omit for TOFU
  --refresh-seconds 86400   # 24h is the default; lower for fast iteration
  --enable                  # only if this is the user's first URL source

Catalogs validated against the same schema/v1.json shipped here load cleanly into any devboy ≥ 0.27. Consumers see the URL-source entries as url origin in devboy secrets catalog status, alongside a [pin:<sha8>…] or [tofu] marker for the trust state.

A separate canonical reference repo is tracked as issue #258 (token-reference repo + crawler) — that effort owns the markdown-first procedure docs (where to obtain / rotate / revoke each token) plus a scheduled crawler to keep them fresh. The catalog JSONs documented here can live alongside or in their own repo; both are first-class.

#Token catalog: how to author your own

#Why this exists

#Discovery sources

#File format

#Required fields

#Optional fields

#Authoring a new provider

#Authoring a new variant for an existing provider

#Worked example

#What the GUI does with it

#Stability promise

#Sharing catalogs across a team or community

#Recommended layout for a community / team catalog repo

#See also