lhumina_code/hero_proc

Fork 0

secrets management #36

New issue

Closed

opened 2026-04-04 08:00:56 +00:00 by despiegk · 4 comments

despiegk commented

2026-04-04 08:00:56 +00:00

Owner

in HERO PROC we manage secrets

FORGE_TOKEN

all to set the FORGE_TOKEN
this token is needed to get info from FORGE on our behalf

check if we can get to FORGE

more info how all of this works see

/Volumes/T7/code0/hero_lib/crates/tools/README.md

pull secrets

implement function pull_secrets()

do as we do in /Users/despiegk/hero/code/hero_skills/tools/modules/secrets_lib.nu
THIS IS IMPLEMENTED IN /Volumes/T7/code0/hero_lib/crates/tools
use this library, check all functionality is there

only do this if FORGE_TOKEN set ,otherwise error

load all the secrets from remote and put them in local secrets

do init secrets at end (maybe there are SECRETS in ENV which are not on FORGE yet)

push secrets

all known secrets from hero_proc are pushed to that location on FORGE (default location), create repo if needed
if repo is there, then get the info of what is there are give prio to info we have in hero_proc
the the most complete version is saved remotely

use functionality from /Volumes/T7/code0/hero_lib/crates/tools or add it to it

init secrets

in admin put a "init secrets" action which calls a function

init_secrets() which is in openrpc and server in submodule for secrets mgmt

reach which KEYS we have in SECRETS from /Users/despiegk/hero/code/hero_skills/tools/modules/secrets_lib.nu (all in upper case)

load the env variables as mentioned there, start a subprocess check if we can find these variables in EXPORT'ed ENV variables from OS, if so and not in SECRETS YET in our hero_proc then put them there

the priority is secrets already loaded, don't overwrite

use /Volumes/T7/code0/hero_lib/crates/tools, if there, if not add it

in HERO PROC we manage secrets ## FORGE_TOKEN all to set the FORGE_TOKEN this token is needed to get info from FORGE on our behalf check if we can get to FORGE more info how all of this works see /Volumes/T7/code0/hero_lib/crates/tools/README.md ## pull secrets implement function pull_secrets() do as we do in /Users/despiegk/hero/code/hero_skills/tools/modules/secrets_lib.nu THIS IS IMPLEMENTED IN /Volumes/T7/code0/hero_lib/crates/tools use this library, check all functionality is there only do this if FORGE_TOKEN set ,otherwise error load all the secrets from remote and put them in local secrets do init secrets at end (maybe there are SECRETS in ENV which are not on FORGE yet) ## push secrets all known secrets from hero_proc are pushed to that location on FORGE (default location), create repo if needed if repo is there, then get the info of what is there are give prio to info we have in hero_proc the the most complete version is saved remotely use functionality from /Volumes/T7/code0/hero_lib/crates/tools or add it to it ## init secrets in admin put a "init secrets" action which calls a function init_secrets() which is in openrpc and server in submodule for secrets mgmt reach which KEYS we have in SECRETS from /Users/despiegk/hero/code/hero_skills/tools/modules/secrets_lib.nu (all in upper case) load the env variables as mentioned there, start a subprocess check if we can find these variables in EXPORT'ed ENV variables from OS, if so and not in SECRETS YET in our hero_proc then put them there the priority is secrets already loaded, don't overwrite use /Volumes/T7/code0/hero_lib/crates/tools, if there, if not add it

despiegk commented

2026-04-04 08:06:03 +00:00

Author

Owner

Implementation Spec for Issue #36 — Secrets Management

Objective

Add full secrets lifecycle management to hero_proc: setting the FORGE_TOKEN, pulling secrets from a remote Forgejo repository into the local hero_proc secrets store, pushing local secrets to the remote, and initializing secrets from OS environment variables. All operations are exposed via OpenRPC (JSON-RPC 2.0) and wired into the admin UI.

Requirements

FORGE_TOKEN: Ability to set FORGE_TOKEN in hero_proc secrets, validate connectivity to forge
pull_secrets(): Load all secrets from the remote Forgejo {username}/secrets repo into hero_proc's SQLite secrets store; requires FORGE_TOKEN; call init_secrets() at the end
push_secrets(): Push all hero_proc secrets to the remote Forgejo {username}/secrets repo; create repo if needed; merge with remote giving priority to local values
init_secrets(): Read canonical secret key names from the template; scan OS environment variables; import found env vars into hero_proc secrets (do not overwrite existing)
All three functions exposed as OpenRPC methods: secret.pull, secret.push, secret.init
Admin UI gets an "Init Secrets" action button

Architecture

The existing hero_proc secrets infrastructure provides SQLite-backed CRUD (secret.set/get/list/delete). The herolib_tools crate in hero_lib provides:

secrets::source() — load secrets.toml as a flat HashMap
secrets::SecretsSync — clone/pull the secrets repo from Forgejo, merge template keys
secrets::SECRETS_TEMPLATE — canonical list of all secret key names
secrets::secrets_flatten() — flatten TOML sections into uppercase key map
secrets::secrets_set_var() — update a single key in secrets.toml
forge::ForgeClient — Forgejo API (create_repo, push, pull, is_online, user)

Plan: add herolib_tools as a dependency to hero_proc_server and implement three new RPC handlers that bridge between file-based secrets (herolib_tools) and SQLite-backed secrets (hero_proc_lib).

Implementation Plan

Step 1: Add herolib_tools dependency (no deps)

Add herolib_tools to workspace and server crate Cargo.toml

Step 2: Implement init_secrets handler (depends on Step 1)

Read SECRETS_TEMPLATE keys, check OS env vars, import into DB if missing

Step 3: Implement pull_secrets handler (depends on Steps 1, 2)

Validate FORGE_TOKEN, clone/pull remote secrets, load into DB, call init at end

Step 4: Implement push_secrets handler (depends on Step 1)

Validate FORGE_TOKEN, create repo if needed, merge local secrets into TOML, push

Step 5: Wire handlers into RPC dispatch (depends on Steps 2-4)

Add secret.init, secret.pull, secret.push to dispatch match

Step 6: Update OpenRPC discovery schema (depends on Step 5)

Add method definitions to rpc.discover

Step 7: Add Init Secrets button to admin UI (depends on Step 5)

Add button that calls secret.init via /rpc proxy

Acceptance Criteria

secret.init reads canonical keys from template, imports env vars not already in DB
secret.pull fails with clear error if FORGE_TOKEN is not set
secret.pull clones/pulls remote secrets repo, loads into hero_proc DB, calls init at end
secret.push creates repo on forge if it does not exist
secret.push merges hero_proc secrets into secrets.toml (local priority), pushes
rpc.discover includes the three new methods
Admin UI has "Init Secrets" button
Forge/git operations run inside spawn_blocking
Existing secret.set/get/list/delete unchanged

Notes

herolib_tools uses synchronous ureq HTTP — all forge/git ops wrapped in spawn_blocking
FORGE_TOKEN can come from hero_proc DB or OS environment
Merge strategy for push: hero_proc values win, remote-only keys preserved

## Implementation Spec for Issue #36 — Secrets Management ### Objective Add full secrets lifecycle management to hero_proc: setting the FORGE_TOKEN, pulling secrets from a remote Forgejo repository into the local hero_proc secrets store, pushing local secrets to the remote, and initializing secrets from OS environment variables. All operations are exposed via OpenRPC (JSON-RPC 2.0) and wired into the admin UI. ### Requirements - **FORGE_TOKEN**: Ability to set FORGE_TOKEN in hero_proc secrets, validate connectivity to forge - **pull_secrets()**: Load all secrets from the remote Forgejo `{username}/secrets` repo into hero_proc's SQLite secrets store; requires FORGE_TOKEN; call init_secrets() at the end - **push_secrets()**: Push all hero_proc secrets to the remote Forgejo `{username}/secrets` repo; create repo if needed; merge with remote giving priority to local values - **init_secrets()**: Read canonical secret key names from the template; scan OS environment variables; import found env vars into hero_proc secrets (do not overwrite existing) - All three functions exposed as OpenRPC methods: `secret.pull`, `secret.push`, `secret.init` - Admin UI gets an "Init Secrets" action button ### Architecture The existing hero_proc secrets infrastructure provides SQLite-backed CRUD (secret.set/get/list/delete). The `herolib_tools` crate in hero_lib provides: - `secrets::source()` — load secrets.toml as a flat HashMap - `secrets::SecretsSync` — clone/pull the secrets repo from Forgejo, merge template keys - `secrets::SECRETS_TEMPLATE` — canonical list of all secret key names - `secrets::secrets_flatten()` — flatten TOML sections into uppercase key map - `secrets::secrets_set_var()` — update a single key in secrets.toml - `forge::ForgeClient` — Forgejo API (create_repo, push, pull, is_online, user) Plan: add `herolib_tools` as a dependency to `hero_proc_server` and implement three new RPC handlers that bridge between file-based secrets (herolib_tools) and SQLite-backed secrets (hero_proc_lib). ### Implementation Plan **Step 1: Add herolib_tools dependency** (no deps) - Add `herolib_tools` to workspace and server crate Cargo.toml **Step 2: Implement init_secrets handler** (depends on Step 1) - Read SECRETS_TEMPLATE keys, check OS env vars, import into DB if missing **Step 3: Implement pull_secrets handler** (depends on Steps 1, 2) - Validate FORGE_TOKEN, clone/pull remote secrets, load into DB, call init at end **Step 4: Implement push_secrets handler** (depends on Step 1) - Validate FORGE_TOKEN, create repo if needed, merge local secrets into TOML, push **Step 5: Wire handlers into RPC dispatch** (depends on Steps 2-4) - Add secret.init, secret.pull, secret.push to dispatch match **Step 6: Update OpenRPC discovery schema** (depends on Step 5) - Add method definitions to rpc.discover **Step 7: Add Init Secrets button to admin UI** (depends on Step 5) - Add button that calls secret.init via /rpc proxy ### Acceptance Criteria - [ ] `secret.init` reads canonical keys from template, imports env vars not already in DB - [ ] `secret.pull` fails with clear error if FORGE_TOKEN is not set - [ ] `secret.pull` clones/pulls remote secrets repo, loads into hero_proc DB, calls init at end - [ ] `secret.push` creates repo on forge if it does not exist - [ ] `secret.push` merges hero_proc secrets into secrets.toml (local priority), pushes - [ ] `rpc.discover` includes the three new methods - [ ] Admin UI has "Init Secrets" button - [ ] Forge/git operations run inside `spawn_blocking` - [ ] Existing secret.set/get/list/delete unchanged ### Notes - `herolib_tools` uses synchronous `ureq` HTTP — all forge/git ops wrapped in `spawn_blocking` - FORGE_TOKEN can come from hero_proc DB or OS environment - Merge strategy for push: hero_proc values win, remote-only keys preserved

despiegk commented

2026-04-04 08:19:32 +00:00

Author

Owner

Test Results

Status: PASS
Details: All 294 tests passed (0 failed, 16 ignored). Breakdown: 8 integration tests, 157 hero_proc_lib unit tests, 18 SDK tests, 55 server tests, 4 CLI integration tests, 6 doc-tests (all ignored). Build and test completed successfully.

Automated test run

## Test Results - Status: PASS - Details: All 294 tests passed (0 failed, 16 ignored). Breakdown: 8 integration tests, 157 hero_proc_lib unit tests, 18 SDK tests, 55 server tests, 4 CLI integration tests, 6 doc-tests (all ignored). Build and test completed successfully. _Automated test run_

despiegk commented

2026-04-04 08:20:14 +00:00

Author

Owner

Implementation Summary

Changes Made

New RPC methods (crates/hero_proc_server/src/rpc/secret.rs):

secret.init — Scans OS environment variables for all canonical secret keys (from the full nushell template: AI, DB, CFG, Services sections — 50+ keys). Imports any found env vars into the hero_proc SQLite secrets DB without overwriting existing values.
secret.pull — Validates FORGE_TOKEN, gets Forgejo username, clones/pulls {username}/secrets repo to ~/hero/code/secrets, loads all non-empty secrets from secrets.toml into DB, then runs init to capture any additional env vars.
secret.push — Validates FORGE_TOKEN, creates {username}/secrets repo on Forge if it doesn't exist, clones/pulls locally, merges all hero_proc DB secrets into secrets.toml (local values take priority), commits and pushes to remote.

RPC dispatch (crates/hero_proc_server/src/rpc/mod.rs):

Wired secret.init, secret.pull, secret.push with async .await (handlers use spawn_blocking for I/O)

OpenRPC spec (crates/hero_proc_server/openrpc.json):

Added method definitions for all three new methods with typed result schemas

Admin UI (crates/hero_proc_ui/templates/index.html, static/js/dashboard.js):

Added "Secrets Management" section in Admin tab with three buttons: Init Secrets, Pull Secrets, Push Secrets
Each button calls the corresponding RPC method and shows toast with results

Dependencies (Cargo.toml, crates/hero_proc_server/Cargo.toml):

Added ureq (HTTP client for Forgejo API), toml (TOML parsing), dirs (home directory resolution) as workspace and server dependencies

Architecture Decisions

Implemented directly in hero_proc_server (no herolib_tools dependency) to avoid circular dependency via herolib_tools → hero_proc_sdk
Uses same patterns as the nushell reference (secrets_lib.nu) and Rust herolib_tools library
All blocking I/O (git commands, HTTP to Forgejo) wrapped in tokio::task::spawn_blocking

Test Results

All 294 tests pass, 0 failures

## Implementation Summary ### Changes Made **New RPC methods** (`crates/hero_proc_server/src/rpc/secret.rs`): - `secret.init` — Scans OS environment variables for all canonical secret keys (from the full nushell template: AI, DB, CFG, Services sections — 50+ keys). Imports any found env vars into the hero_proc SQLite secrets DB without overwriting existing values. - `secret.pull` — Validates FORGE_TOKEN, gets Forgejo username, clones/pulls `{username}/secrets` repo to `~/hero/code/secrets`, loads all non-empty secrets from `secrets.toml` into DB, then runs init to capture any additional env vars. - `secret.push` — Validates FORGE_TOKEN, creates `{username}/secrets` repo on Forge if it doesn't exist, clones/pulls locally, merges all hero_proc DB secrets into `secrets.toml` (local values take priority), commits and pushes to remote. **RPC dispatch** (`crates/hero_proc_server/src/rpc/mod.rs`): - Wired `secret.init`, `secret.pull`, `secret.push` with async `.await` (handlers use `spawn_blocking` for I/O) **OpenRPC spec** (`crates/hero_proc_server/openrpc.json`): - Added method definitions for all three new methods with typed result schemas **Admin UI** (`crates/hero_proc_ui/templates/index.html`, `static/js/dashboard.js`): - Added "Secrets Management" section in Admin tab with three buttons: Init Secrets, Pull Secrets, Push Secrets - Each button calls the corresponding RPC method and shows toast with results **Dependencies** (`Cargo.toml`, `crates/hero_proc_server/Cargo.toml`): - Added `ureq` (HTTP client for Forgejo API), `toml` (TOML parsing), `dirs` (home directory resolution) as workspace and server dependencies ### Architecture Decisions - Implemented directly in hero_proc_server (no herolib_tools dependency) to avoid circular dependency via `herolib_tools → hero_proc_sdk` - Uses same patterns as the nushell reference (`secrets_lib.nu`) and Rust `herolib_tools` library - All blocking I/O (git commands, HTTP to Forgejo) wrapped in `tokio::task::spawn_blocking` ### Test Results - All 294 tests pass, 0 failures