Crypto Package Consolidation & Improvements #46

Merged

thabeta merged 3 commits from develop-crypto-consolidation into develop

2026-01-08 16:02:06 +00:00

sameh-farouk commented

2026-01-08 15:45:18 +00:00

Member

Overview

This PR consolidates the herolib-crypto package into herolib-crypt, establishing a single source of truth for all cryptographic operations in the ecosystem. It eliminates code duplication, unifies testing pathways, and improves the internal architecture while maintaining 100% backward compatibility for downstream consumers (vault, Rhai scripts, etc.).

Why Consolidation?

Previously, we maintained two parallel crypto packages (crypt and crypto) with redundant signing logic and divergent Rhai bindings. This consolidation:

Eliminates Code duplication: Removes redundant Ed25519 signing implementations.
Centralizes Security: Ensures key management (zeroization, formatting) is handled in one audited location.
Unifies Tooling: Merges test runners and examples into a cohesive set.

Architecture: The `keys` Module

We have adopted the keys module (from the original crypto package) as the core internal implementation.

Legacy Signing Removal

We removed the old procedural signing logic in asymmetric/signing.rs that relied on raw byte arrays. This old implementation was prone to errors (e.g., incorrect slice lengths) and lacked the robust error handling of the keys module. The signing.rs file now acts as a thin, backward-compatible wrapper delegating all logic to the keys module.

Why `keys`?

Type Safety: Operations use distinct types (Ed25519Keypair, Ed25519PublicKey) rather than raw bytes, preventing misuse.
Object-Oriented: Logically groups operations (keypair.sign(), pubkey.verify()).
Security: Centralizes sensitive data handling (e.g., Zeroize support) and ensures consistent hex/byte serialization.

API Preservation

We maintained strict backward compatibility and added:

Rhai API

Keys Module (30 functions):

ed25519_generate(), ed25519_from_hex(), ed25519_from_bytes()
sign(), sign_bytes(), verify(), verify_bytes()
public_key(), public_key_hex(), public_key_bytes()
to_hex(), to_bytes() (for keypair, pubkey, signature)
random_bytes(), encode_hex(), decode_hex()
sha256(), sha256_bytes(), secure_compare()
verify_ed25519(), verify_ed25519_bytes()

Important note: Safety vs Convenience

Rhai String API (asymmetric module): Less safe. Rhai strings are immutable but linger in memory. You pass the sensitive private key string around by value.
New Rhai Object API (keys module):
- Safer. The Ed25519Keypair type in Rhai wraps the Rust struct. We can control how it's printed (hiding the key) and potentially how it's dropped (zeroizing). It encapsulates the secret better than a raw string.
- Faster. This is better if you need to use that key multiple times (e.g., signing 5 different headers). You parse once, use many times.

We keep both API in Rhai because:

Old scripts must continue to run (String API).
Simple scripts should remain simple (String API).
Complex scripts need state, better performance, and object-oriented features (Object API).

HTTP Signatures Module (10 functions):

httpsig_signer_new(), httpsig_signer_with_headers(), httpsig_signer_with_label()
httpsig_sign(), httpsig_verifier_new(), httpsig_verifier_with_key()
httpsig_verifier_with_tolerance(), httpsig_verify()
httpsig_compute_digest(), httpsig_extract_key_id()

Rust API

Module	Description
`crypt::asymmetric`	High-level dual keypair (Ed25519 + X25519)
`crypt::symmetric`	Password-based encryption (XChaCha20-Poly1305)
`crypt::keys`	Low-level Ed25519 primitives
`crypt::httpsig`	RFC 9421 HTTP Message Signatures

Architecture Comparison: `asymmetric` vs `keys`

We intentionally preserve both modules to serve different needs:

Feature	`crypt::asymmetric` (High-Level)	`crypt::keys` (Low-Level Primitive)
Key Composition	Dual Keypair: Generates 4 keys per identity (Ed25519 for signing + X25519 for encryption).	Single Keypair: Strictly Ed25519 signing keys only.
Primary Use Case	User Identities: Perfect for user accounts that need both signing identities and message encryption capabilities.	System Primitives: Ideal for lightweight signing needs (e.g., HTTP Signatures, tokens) where encryption unused.
Data Types	Strings: Input/Output is universally hex strings. Easy to use, but requires parsing validation on every call.	Strong Types: Uses `Ed25519Keypair` / `Signature` structs. Zero-cost validation after creation.
Overhead	Higher: Generates unused X25519 keys if you only need signing.	Minimal: Generates exactly what you need.

Improvements

Downstream Integration

herolib-do (Herodo): Updated REPL with a new /crypt command to expose all crypto functions with autocomplete and help documentation.
herolib-vault: Compiled and verified against the new herolib-crypt core.

Tooling

Unified Runner: Consolidated run_rhai and rhai_runner into a single, smart tool (examples/run_rhai.rs).
- Run all: cargo run --example run_rhai
- Run one: cargo run --example run_rhai -- path/to/script.rhai

Feature Flags

[features]
default = ["httpsig"]
httpsig = ["dep:http"]
full = ["httpsig", "rhai"]

README Updates

The README was updated:

Added new Keys and HTTP Signatures documentation
Updated module structure diagram

Verification

Category	Status
Unit Tests	32 passed
Doc Tests	33 passed
Keys Rhai Tests	6/6 passed
Httpsig Rhai Tests	7/7 passed
API Compatibility	100% preserved

Deleted

Entire packages/crypto/ directory
Misplaced vault-specific Rhai tests (keypair/*.rhai)

## Overview This PR consolidates the `herolib-crypto` package into `herolib-crypt`, establishing a **single source of truth** for all cryptographic operations in the ecosystem. It eliminates code duplication, unifies testing pathways, and improves the internal architecture while maintaining 100% backward compatibility for downstream consumers (vault, Rhai scripts, etc.). ## Why Consolidation? Previously, we maintained two parallel crypto packages (`crypt` and `crypto`) with redundant signing logic and divergent Rhai bindings. This consolidation: 1. **Eliminates Code duplication**: Removes redundant Ed25519 signing implementations. 2. **Centralizes Security**: Ensures key management (zeroization, formatting) is handled in one audited location. 3. **Unifies Tooling**: Merges test runners and examples into a cohesive set. ## Architecture: The `keys` Module We have adopted the `keys` module (from the original `crypto` package) as the **core internal implementation**. ### Legacy Signing Removal We removed the old procedural signing logic in `asymmetric/signing.rs` that relied on raw byte arrays. This old implementation was prone to errors (e.g., incorrect slice lengths) and lacked the robust error handling of the `keys` module. The `signing.rs` file now acts as a **thin, backward-compatible wrapper** delegating all logic to the `keys` module. ### Why `keys`? - **Type Safety**: Operations use distinct types (`Ed25519Keypair`, `Ed25519PublicKey`) rather than raw bytes, preventing misuse. - **Object-Oriented**: Logically groups operations (`keypair.sign()`, `pubkey.verify()`). - **Security**: Centralizes sensitive data handling (e.g., `Zeroize` support) and ensures consistent hex/byte serialization. ## API Preservation We maintained strict backward compatibility and added: ### Rhai API **Keys Module (30 functions):** - `ed25519_generate()`, `ed25519_from_hex()`, `ed25519_from_bytes()` - `sign()`, `sign_bytes()`, `verify()`, `verify_bytes()` - `public_key()`, `public_key_hex()`, `public_key_bytes()` - `to_hex()`, `to_bytes()` (for keypair, pubkey, signature) - `random_bytes()`, `encode_hex()`, `decode_hex()` - `sha256()`, `sha256_bytes()`, `secure_compare()` - `verify_ed25519()`, `verify_ed25519_bytes()` **Important note: Safety vs Convenience** - Rhai String API (`asymmetric` module): Less safe. Rhai strings are immutable but linger in memory. You pass the sensitive private key string around by value. - New Rhai Object API (`keys` module): - Safer. The Ed25519Keypair type in Rhai wraps the Rust struct. We can control how it's printed (hiding the key) and potentially how it's dropped (zeroizing). It encapsulates the secret better than a raw string. - Faster. This is better if you need to use that key multiple times (e.g., signing 5 different headers). You parse once, use many times. We keep both API in Rhai because: * Old scripts must continue to run (String API). * Simple scripts should remain simple (String API). * Complex scripts need state, better performance, and object-oriented features (Object API). **HTTP Signatures Module (10 functions):** - `httpsig_signer_new()`, `httpsig_signer_with_headers()`, `httpsig_signer_with_label()` - `httpsig_sign()`, `httpsig_verifier_new()`, `httpsig_verifier_with_key()` - `httpsig_verifier_with_tolerance()`, `httpsig_verify()` - `httpsig_compute_digest()`, `httpsig_extract_key_id()` ### Rust API | Module | Description | |--------|-------------| | `crypt::asymmetric` | High-level dual keypair (Ed25519 + X25519) | | `crypt::symmetric` | Password-based encryption (XChaCha20-Poly1305) | | `crypt::keys` | Low-level Ed25519 primitives | | `crypt::httpsig` | RFC 9421 HTTP Message Signatures | ## Architecture Comparison: `asymmetric` vs `keys` We intentionally preserve both modules to serve different needs: | Feature | `crypt::asymmetric` (High-Level) | `crypt::keys` (Low-Level Primitive) | | :--- | :--- | :--- | | **Key Composition** | **Dual Keypair**: Generates 4 keys per identity (Ed25519 for signing + X25519 for encryption). | **Single Keypair**: Strictly Ed25519 signing keys only. | | **Primary Use Case** | **User Identities**: Perfect for user accounts that need both signing identities and message encryption capabilities. | **System Primitives**: Ideal for lightweight signing needs (e.g., HTTP Signatures, tokens) where encryption unused. | | **Data Types** | **Strings**: Input/Output is universally hex strings. Easy to use, but requires parsing validation on every call. | **Strong Types**: Uses `Ed25519Keypair` / `Signature` structs. Zero-cost validation after creation. | | **Overhead** | **Higher**: Generates unused X25519 keys if you only need signing. | **Minimal**: Generates exactly what you need. | ## Improvements ### Downstream Integration - **`herolib-do` (Herodo)**: Updated REPL with a new `/crypt` command to expose all crypto functions with autocomplete and help documentation. - **`herolib-vault`**: Compiled and verified against the new `herolib-crypt` core. ### Tooling - **Unified Runner**: Consolidated `run_rhai` and `rhai_runner` into a single, smart tool (`examples/run_rhai.rs`). - Run all: `cargo run --example run_rhai` - Run one: `cargo run --example run_rhai -- path/to/script.rhai` ## Feature Flags ```toml [features] default = ["httpsig"] httpsig = ["dep:http"] full = ["httpsig", "rhai"] ``` ## README Updates The README was updated: - Added new Keys and HTTP Signatures documentation - Updated module structure diagram ## Verification | Category | Status | |----------|--------| | Unit Tests | 32 passed | | Doc Tests | 33 passed | | Keys Rhai Tests | 6/6 passed | | Httpsig Rhai Tests | 7/7 passed | | API Compatibility | 100% preserved | ## Deleted - Entire `packages/crypto/` directory - Misplaced vault-specific Rhai tests (`keypair/*.rhai`)

sameh-farouk added 3 commits

2026-01-08 15:45:19 +00:00