> ## Documentation Index > Fetch the complete documentation index at: https://continuum-ec12e897.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Running a keeper > Build, configure, deploy, and troubleshoot the reference keeper. The reference keeper is a Rust async bot. Source: [`keeper/`](https://github.com/continuum-markets/continuum/tree/main/keeper) in the protocol repo. This page is for operators who want to run their own. Most integrators don't need to. ## Prerequisites * Rust 1.75+ and Cargo * Solana CLI 1.18+ * A funded keeper wallet (≥ 0.5 SOL on devnet, more on mainnet) * A separate CLP authority keypair (≥ 0.1 SOL) * A private RPC URL (Helius, Triton, etc.) - public devnet is rate-limited * The keeper wallet must be the canonical `market.keeper_authority` for any market you want to operate (single-pubkey constraint today); the CLP authority key signs book shapes and inventory plumbing ## Build ```bash theme={null} git clone https://github.com/continuum-markets/continuum.git cd continuum/keeper cargo build --release ``` Output: `keeper/target/release/keeper` (\~50MB binary). ## Configure The keeper reads a single `keeper/.env` (override the path with `KEEPER_ENV_FILE`). Create it with the keys below — these are the knobs the keeper actually consumes: ```env theme={null} # RPC - set both for dual-network mode MAINNET_RPC_URL=https://mainnet.helius-rpc.com/?api-key=YOUR_KEY DEVNET_RPC_URL=https://devnet.helius-rpc.com/?api-key=YOUR_KEY # Keypairs - JSON byte array (Railway) or file path (local) KEEPER_KEYPAIR_JSON=[12,34,56,...] # or KEEPER_KEYPAIR=~/.config/solana/id.json CLP_AUTHORITY_KEYPAIR_JSON=[...] # separate from KEEPER_KEYPAIR # or CLP_AUTHORITY_KEYPAIR=keeper/key/keeper-keypair.json # USDC - ON DEVNET, MUST SET EXPLICITLY DEVNET_USDC_MINT=B1c5xBYkp7AAemYhcu4VuH4CU4sPJDDuG2iuv6ts38uE MAINNET_USDC_MINT=EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v # Active markets ACTIVE_MARKETS=QQQ,SPY,XAU,VXX # Logging RUST_LOG=continuum_keeper=info ``` ### Kill switches (all default ON) Every keeper behavior is an independent task behind an env gate, so any loop can be disabled without a redeploy: | Variable | Gates | | ---------------------------- | ---------------------------------------------------------------------------- | | `CLMM_POOLS_ENABLED` | The book manager (reposition + shape posting) | | `CLMM_VOL_SPREAD_ENABLED` | Vol-scaled spreads (off = static geometry) | | `CLMM_REFILL_ENABLED` | Inventory plumbing (refill / trim / bid cap) | | `CLMM_BUFFER_POST_ENABLED` | Dynamic overcollateralization target posting | | `CLMM_REBALANCE_ENABLED` | Cross-market collateral rebalance | | `CLMM_FEE_SCALE_ENABLED` | Vol-informed mint/redeem fee posting | | `CLMM_COMMIT_SPREAD_ENABLED` | Vol-scaled committed-order spread posting (`settle_swap` floors it at 10bps) | | `COMMIT_SETTLE_ENABLED` | The committed-order settlement cranker | | `DEVNET_YIELD_EMULATION` | Devnet yield emulator (hard devnet-only gate in code) | Off by default (opt-in): | Variable | Gates | | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `COLLATERAL_YIELD_ENABLED` | Deploy idle collateral to lending venues at all (default **off**) | | `COLLATERAL_YIELD_RATE_AWARE` | Use the rate-impact "water-fill" allocator that splits across venues to equalize marginal rates (default **on**, only acts when yield is enabled and ≥2 venues report TVL) | | `CLMM_GAMMA_DECAY_ENABLED` | Post the short-leg volatility decay (`set_s_gamma_index`); default **off** until the on-chain ix is deployed + Markets realloc'd | | `GAMMA_DECAY_COEF` | Multiplier on the per-cycle decay; default `1.0` = the r39-calibrated value (decay = realized variance) | ### Tunables | Variable | Default | Meaning | | --------------------------------------------------- | ----------- | --------------------------------------------------------------------------------------------- | | `CLMM_REPOSITION_BPS` / `CLMM_REPOSITION_BPS_` | 20 (VXX 40) | Drift gate before a book re-pins to NAV | | `CLMM_BID_TARGET_USD` | 5000 | Standing bid cUSDC per side; excess sweeps to the CLP vault (idle bids forgo lending carry) | | `CLMM_TRIM_RATIO` | 1.5 | Max ask value as a multiple of bid value; excess synth sweeps back to CLP custody | | `COMMIT_SETTLE_INTERVAL_SECS` | 20 | Settler cadence when idle (drops to 5s while orders are pending) | | `HERMES_STALE_SECS` | 120 | Feed publish-time age beyond which the oracle push is withheld (closed-market detection) | | `DEVNET_YIELD_APR` | 0.055 | Emulator fallback APR when the DefiLlama cache is cold (live top-venue APY is used when warm) | **`USDC_MINT` is the most common misconfig.** If unset, it falls back to mainnet USDC even on devnet - this silently breaks every ATA derivation, vault balance reads as 0, and the seeder skips with no error. Always set `DEVNET_USDC_MINT=B1c5xBYkp7AAemYhcu4VuH4CU4sPJDDuG2iuv6ts38uE` on devnet. ## Run ```bash theme={null} ./target/release/keeper ``` Or via the workspace script: ```bash theme={null} cd .. # back to repo root bun run keeper:local ``` You should see, within \~30 seconds: ``` INFO continuum_keeper: starting keeper INFO continuum_keeper::oracle: pushed observation for QQQ price=480.32 conf=0.04 INFO continuum_keeper::risk: market QQQ state=Normal INFO continuum_keeper::dashboard: http listener on 0.0.0.0:8485 ``` The dashboard is at `http://localhost:8485` (devnet) or `:8484` (mainnet). ## Multi-keeper considerations The market's `keeper_authority` is a single pubkey. Two keepers signing as the same pubkey will collide on `keeper_*_single` calls - only one tx will land per slot. If you want redundancy: * **Active/standby**: peer heartbeat (already in the keeper) + manual primary/standby logic. The reference keeper has basic peer heartbeat; primary/standby logic is on the [roadmap](/markets/roadmap). * **Different markets per operator**: rotate `keeper_authority` per market. `update_keeper_authority` is admin-callable. External bots (no keeper authority) can run alongside the canonical keeper without coordination - committed-order settlement is permissionless, and anyone can take residual paired-arb opportunities. ## Solvency-checking the keeper ```bash theme={null} # Check keeper wallet balance solana balance # Check L/S residual in keeper wallet (should be 0 in steady state) spl-token accounts --owner # Check global vault balance bun run scripts/protocol-status.ts ``` If you see non-zero L or S in the keeper wallet, restart the keeper - boot-time sweep will redeem them. ## Common issues ### "Keeper skipping silently / balances read as 0" The most common issue. Check: 1. Is `USDC_MINT` set correctly for your cluster? (See the warning above - mainnet fallback breaks every ATA derivation.) 2. Is the keeper authority correct? Check `market.keeper_authority` matches your `KEEPER_KEYPAIR` pubkey. 3. Are both wallets funded? Pool inits and refills pay rent from the **CLP authority**, not the keeper wallet - check both balances on insufficient-lamports errors. ### "Unauthorized keeper" The keeper signed with a key that doesn't match `market.keeper_authority`. Two common causes: 1. Wrong `KEEPER_KEYPAIR` configured. 2. Admin rotated `keeper_authority` and you didn't update. Check the on-chain market account vs your keypair pubkey. ### "RPC rate-limited" The keeper has fallback RPC built in - set a comma-separated list: ```env theme={null} MAINNET_RPC_URL=https://primary,https://fallback ``` The keeper rotates on 429. For production, run a dedicated RPC (Helius / Triton premium tier). ### "Keeper wallet has stranded synth tokens" Restart the keeper. Boot-time sweep handles it. If the keeper can't run, manually call `redeem_paired` (paired residue) or `keeper_redeem_single` (asymmetric) signed with the keeper authority. ### "Book prices drifted far from NAV" First check whether the underlying market is simply **closed**: the oracle withholds pushes on a stale feed, mint/redeem self-freezes, and the books float at their standing band by design - the first fresh print re-anchors them. If the market is open: 1. **Keeper not repositioning** → check `CLMM_POOLS_ENABLED` and the book-manager logs (`continuum_keeper::clmm=debug`). 2. **Pool depth exhausted** → the refill loop should restock within a cycle; check `CLMM_REFILL_ENABLED` and CLP vault balances. 3. **Oracle stale while market open** → check the Hermes feed and `HERMES_STALE_SECS`. ## Production deployment notes * **Multisig the admin keypair.** The keeper authority is a hot key by design (signs every cycle). The admin authority should be a Squads multisig - separate from the keeper. * **Run one keeper per cluster.** The reference keeper supports dual-network mode (one process, both mainnet and devnet). Separate dashboard ports. * **Monitor the dashboard.** It's on `:8484` / `:8485`. Pipe to your monitoring (Datadog, Grafana, custom Prometheus exporter). * **Set up alerts.** Watch for: stale oracle while markets are open (> 60s), pending committed orders aging toward TTL, low SOL balance on either signer (\< 0.1). * **Keep both signers topped up.** The keeper wallet pays oracle pushes and settles; the CLP authority pays book repositions and inventory plumbing. ## Logging ```env theme={null} RUST_LOG=continuum_keeper=info,continuum_keeper::clmm=debug ``` Per-module log levels. Useful in production: * `continuum_keeper=info` - normal operation * `continuum_keeper::clmm=debug` - every book reposition and refill decision * `continuum_keeper::commit=debug` - committed-order discovery and settlement * `continuum_keeper::rpc=warn` - RPC failures only ## Updating ```bash theme={null} cd continuum git pull cd keeper cargo build --release # Restart with: systemctl restart continuum-keeper # or whatever your supervisor is ``` The keeper does a clean shutdown (boot-time sweep is idempotent so a fast restart is safe). ## See also What the keeper does, why it's privileged. Why v2 self-pegs and what bounds drift. Reposition loop, refill, trim, bid cap.