Hero Redis

A high-performance, Redis-compatible server built on redb with ChaCha20-Poly1305 encryption and dual authentication modes.

Features

• Redis Protocol Compatible - Works with any Redis client including Redis Desktop Manager
• Dual Authentication:
  • Simple: Secret-based auth with AUTH <secret> (recommended for most users)
  • Advanced: Ed25519 signature-based auth (for high-security environments)
• Multi-Database with ACL - Per-database Read/Write/Admin permissions
• Persistent Storage - Data stored in redb (pure Rust embedded database)
• Encryption - All values encrypted with ChaCha20-Poly1305
• Multiple Databases - Up to 1000 databases with lazy loading
• Low Memory - ~3MB at startup, databases loaded on demand
• Auto-cleanup - Idle databases automatically closed after 5 minutes
• Unix Socket & TCP - Both connection methods supported
• Vector Search - HNSW-based similarity search with hannoy library
• Cross-platform - Linux (x86_64, aarch64) and macOS (x86_64, aarch64)

Quick Start

Install Pre-built Binaries

The easiest way to get started is to install pre-built binaries:

curl -sSL https://forge.ourworld.tf/geomind_code/hero_redis/raw/branch/main/scripts/install.sh | bash

This installs all tools to ~/hero/bin/:

• hero_redis - The Redis-compatible server
• hero_redis_client - Client with Rhai scripting support
• hero_redis_login - Ed25519 keypair generator
• hero_redis_tester - Integration test tool

Add to your PATH (if not already):

export PATH="$HOME/hero/bin:$PATH"

Supported platforms:

• Linux x86_64 (amd64)
• Linux aarch64 (arm64)
• macOS Apple Silicon (darwin-arm64)

Build from Source

git clone https://forge.ourworld.tf/geomind_code/hero_redis.git
cd hero_redis
cargo build --release --features server

Start Server (Simple Auth)

# Start with admin secret (simplest way)
hero_redis \
  --encryption-key "your-encryption-key" \
  --admin-secret "your-admin-secret" \
  --data-dir ~/.hero_redis

Or if built from source:

./target/release/hero_redis \
  --encryption-key "your-encryption-key" \
  --admin-secret "your-admin-secret"

Connect and Use

# Using redis-cli (default port is 6666)
redis-cli -p 6666

# Authenticate with your secret
> AUTH your-admin-secret
OK

# Now use any Redis command
> SET mykey myvalue
OK
> GET mykey
"myvalue"

Run Rhai Scripts

# Run a single script
hero_redis_client my_script.rhai

# Run all scripts in a directory
hero_redis_client scripts/

Generate Ed25519 Keypair

# Generate a new keypair for Ed25519 authentication
hero_redis_login --generate

Using with Redis Desktop Manager

1. Start server with --admin-secret
2. In Redis Desktop Manager, configure connection:
   • Host: 127.0.0.1
   • Port: 6666
   • Auth: your-admin-secret
3. Connect and use normally

Authentication Modes

1. Simple Auth (Recommended)

Start the server with --admin-secret:

./target/release/hero_redis \
  --encryption-key "encryption-key" \
  --admin-secret "my-secret-password"

Connect with AUTH:

AUTH my-secret-password

Create users with secrets:

USER.CREATESECRET alice alice-secret-123
USER.GRANT 1 alice write

Users authenticate with:

AUTH alice-secret-123

2. Advanced Auth (Ed25519)

For high-security environments, use Ed25519 signature-based authentication:

# Generate keypair
./target/release/hero_redis_login --generate

# Start server with pubkey
./target/release/hero_redis \
  --encryption-key "encryption-key" \
  --admin-pubkey "a1b2c3d4..."

Authentication flow:

CHALLENGE          → returns random challenge
TOKEN <pubkey> <signature>  → returns session token
AUTH <token>       → authenticates session

Command Line Options

Server (hero_redis)

Option	Default	Description
-d, --data-dir	~/.hero_redis	Database directory
-s, --socket	~/.hero_redis/redis.sock	Unix socket path
-p, --port	6666	TCP port (0 to disable)
--encryption-key	required	Encryption key for DB 0
--admin-secret	-	Admin secret for simple auth
--admin-pubkey	-	Admin public key for Ed25519 auth
-v, --verbose	false	Enable debug logging

Note: At least one of --admin-secret or --admin-pubkey must be provided.

User Management

Create Users (Secret-based)

# As admin, create a user with a secret
USER.CREATESECRET <username> <secret>

# Example
USER.CREATESECRET alice mysecretpassword

Create Users (Ed25519-based)

# As admin, register a user's public key
USER.CREATE <pubkey>

# Example
USER.CREATE a1b2c3d4e5f6...

Grant Permissions

# Grant access to a database
USER.GRANT <db_number> <username_or_pubkey> <read|write|admin>

# Examples
USER.GRANT 1 alice write
USER.GRANT 2 bob read
USER.GRANT 1 a1b2c3d4e5f6... admin

Revoke Permissions

USER.REVOKE <db_number> <username_or_pubkey>

# Example
USER.REVOKE 1 alice

Permission Levels

Level	Can Read	Can Write	Can FLUSHDB	Can USER.GRANT	Can DATABASE.CREATE
read	Yes	No	No	No	No
write	Yes	Yes	No	No	No
admin	Yes	Yes	Yes	Yes (same db)	No
server admin	Yes	Yes	Yes	Yes	Yes

Admin Commands

Server Admin Commands

Command	Description
DATABASE.CREATE <encryption_key>	Create new database, returns db number
DATABASE.STATUS [db|all]	Show database info
DATABASE.PUBLIC <db> <on|off>	Enable/disable public read-only access
USER.CREATE <pubkey>	Register Ed25519 user
USER.CREATESECRET <username> <secret>	Create secret-based user
USER.DELETE <username_or_pubkey>	Delete a user
ADMIN.ADD <pubkey>	Add server admin (Ed25519)
ADMIN.REMOVE <pubkey>	Remove server admin
ADMIN.LIST	List server admins

Public Read-Only Access

You can make a database publicly readable (no authentication required for reads):

# As admin, make database 1 public
DATABASE.PUBLIC 1 on

# Now anyone can read without AUTH
SELECT 1
GET publickey    # Works without authentication
SET key value    # Fails - writes still require auth

Database Admin Commands

Command	Description
USER.GRANT <db> <user> <perm>	Grant permission (read/write/admin)
USER.REVOKE <db> <user>	Revoke permission
FLUSHDB	Clear current database

Supported Redis Commands

Authentication

AUTH, SAUTH, CHALLENGE, TOKEN

String Commands

GET, SET, MGET, MSET, DEL, EXISTS, EXPIRE, TTL, KEYS, SCAN, INCR, INCRBY, DECR, DECRBY, APPEND, STRLEN, GETRANGE, SETNX, SETEX, GETSET, TYPE

Hash Commands

HSET, HGET, HMSET, HMGET, HGETALL, HDEL, HEXISTS, HLEN, HKEYS, HVALS, HINCRBY, HSETNX, HSCAN

List Commands

LPUSH, RPUSH, LPOP, RPOP, LRANGE, LLEN, LINDEX, LSET, LREM

Set Commands

SADD, SREM, SMEMBERS, SISMEMBER, SCARD, SPOP, SUNION, SINTER, SDIFF, SSCAN

Stream Commands

XADD, XLEN, XRANGE, XREVRANGE, XREAD, XINFO, XTRIM, XGROUP, XREADGROUP, XACK, XPENDING

Vector Commands

VECTOR.CREATE, VECTOR.ADD, VECTOR.SEARCH, VECTOR.SEARCHBYID, VECTOR.GET, VECTOR.DEL, VECTOR.BUILD, VECTOR.INFO, VECTOR.LIST, VECTOR.DROP, VECTOR.CLEAR, VECTOR.EXISTS, VECTOR.LEN

Connection Commands

PING, AUTH, SAUTH, SELECT, QUIT, COMMAND

Management Commands

COMPACT, SHUTDOWN, MEMORY USAGE, MEMORY STATS, DBSIZE, FLUSHDB, INFO, CONFIG, CLIENT

Vector Search

Hero Redis includes HNSW-based vector similarity search using the hannoy library. Perfect for AI embeddings, semantic search, and RAG applications.

Quick Start

# Create a vector index (128 dimensions, cosine similarity)
VECTOR.CREATE embeddings 128 METRIC cosine

# Add vectors (JSON format)
VECTOR.ADD embeddings 1 [0.1, 0.2, 0.3, ...]
VECTOR.ADD embeddings 2 [0.4, 0.5, 0.6, ...]

# Build the HNSW index (required before searching)
VECTOR.BUILD embeddings

# Search for similar vectors (returns top 10)
VECTOR.SEARCH embeddings [0.1, 0.2, 0.3, ...] 10

# Search by existing vector ID
VECTOR.SEARCHBYID embeddings 1 10

High-Performance Bulk Insert

For bulk loading, use VECTOR.ADDBATCH with binary format - ~97x faster than individual adds:

Method	Rate	10K vectors
Individual VECTOR.ADD	~220 vec/s	~45 sec
Binary VECTOR.ADDBATCH	~21,000 vec/s	~0.5 sec

Binary format (little-endian):

Header (8 bytes):
  - num_vectors: u32 (4 bytes)
  - dimensions:  u32 (4 bytes)

Per vector:
  - id:     u32 (4 bytes)
  - vector: f32 × dimensions

Rhai Script Example

// Create index
vector_create(r, "embeddings", 128, "cosine");

// Batch add (fast!)
let batch = [];
for id in 1..=1000 {
    batch.push([id, generate_embedding(id)]);
}
vector_add_batch(r, "embeddings", 128, batch);

// Build and search
vector_build(r, "embeddings");
let results = vector_search(r, "embeddings", query_vector, 10);

Rust Client Example

use hero_redis::HeroRedisClient;

let mut client = HeroRedisClient::new("127.0.0.1", 6666, private_key)?;
client.select(1)?;

// Create index
client.vector_create("embeddings", 128, "cosine")?;

// Bulk add with binary format (fast!)
let items: Vec<(u32, Vec<f32>)> = embeddings.into_iter().enumerate()
    .map(|(i, v)| (i as u32, v))
    .collect();
client.vector_add_batch("embeddings", 128, &items)?;

// Build and search
client.vector_build("embeddings")?;
let results = client.vector_search("embeddings", &query, 10)?;

Vector Commands Reference

Command	Description
VECTOR.CREATE <index> <dims> METRIC <cosine|euclidean|manhattan>	Create index
VECTOR.ADD <index> <id> <json_vector>	Add single vector
VECTOR.ADDBATCH <index> <binary_data>	Bulk add (binary format, ~97x faster)
VECTOR.BUILD <index>	Build HNSW graph (required before search)
VECTOR.SEARCH <index> <json_vector> <k>	Find k nearest neighbors
VECTOR.SEARCHBYID <index> <id> <k>	Find neighbors of existing vector
VECTOR.GET <index> <id>	Get vector by ID
VECTOR.DEL <index> <id>	Delete vector
VECTOR.INFO <index>	Get index statistics
VECTOR.LEN <index>	Get vector count
VECTOR.LIST	List all indexes
VECTOR.DROP <index>	Delete index
VECTOR.CLEAR <index>	Remove all vectors

Distance Metrics

Metric	Description	Use Case
cosine	Cosine similarity	Text embeddings (OpenAI, etc.)
euclidean	L2 distance	Image embeddings
manhattan	L1 distance	Sparse vectors

Common Embedding Dimensions

Model	Dimensions
OpenAI text-embedding-ada-002	1536
OpenAI text-embedding-3-small	1536
Cohere embed-english-v3.0	1024
all-MiniLM-L6-v2	384
BGE-base-en	768

See docs/ai_embeddings.md for complete documentation.

Using as a Library

Hero Redis supports feature flags to include only what you need, reducing binary size and compile times.

Feature Flags

Feature	Description	Default
client	Redis client with Ed25519 authentication	Yes
server	Redis-compatible server with redb backend	No
rhai	Rhai scripting support for the client	No
vector	Vector database with HNSW similarity search	No
full	All features enabled	No

Usage Examples

# Client only (default) - minimal dependencies
[dependencies]
hero_redis = "0.2"

# Server only (no client)
[dependencies]
hero_redis = { version = "0.2", default-features = false, features = ["server"] }

# Client with Rhai scripting
[dependencies]
hero_redis = { version = "0.2", features = ["rhai"] }

# Client with vector support
[dependencies]
hero_redis = { version = "0.2", features = ["vector"] }

# Everything
[dependencies]
hero_redis = { version = "0.2", features = ["full"] }

Client Example

use hero_redis::hero_redis_client::HeroRedisClient;

// Connect with Ed25519 private key
let mut client = HeroRedisClient::new("127.0.0.1", 6666, "your_private_key_hex")?;

// Or connect with simple password
let mut client = HeroRedisClient::new_with_password("127.0.0.1", 6666, "your_password")?;

// Or use the builder pattern
let mut client = HeroRedisClient::builder()
    .host("127.0.0.1")
    .port(6666)
    .password("your_password")  // or .private_key("hex")?
    .database(1)
    .connect()?;

client.select(1)?;

// String operations
client.set("name", "Alice")?;
let name = client.get("name")?.unwrap();

// Hashes
client.hset("user:1", "name", "Bob")?;
let user = client.hgetall("user:1")?;

// Lists
client.rpush("queue", "job1")?;
let job = client.lpop("queue")?;

// Sets
client.sadd("tags", "rust")?;
let tags = client.smembers("tags")?;

// Admin operations (create database, manage users)
let db_num = client.admin().create_database("my-encryption-key")?;
client.admin().grant_permission(db_num, &user_pubkey, Permission::Write)?;

Rhai Scripting Example

With features = ["rhai"]:

use hero_redis::hero_redis_rhai::RhaiScriptRunner;

let runner = RhaiScriptRunner::builder()
    .host("127.0.0.1")
    .port(6666)
    .private_key("your_private_key_hex")
    .build()?;

runner.run_script("scripts/my_script.rhai")?;

Example Rhai script:

// Select database
select(r, 1);

// Basic operations
set(r, "counter", "0");
incr(r, "counter");
let val = get(r, "counter");
print(`Counter: ${val}`);

// Hashes
hset(r, "user:1", "name", "Alice");
hset(r, "user:1", "email", "alice@example.com");
let user = hgetall(r, "user:1");
print(user);

Data Storage

• Data directory: ~/.hero_redis/ (configurable)
• Database files: db0.redb, db1.redb, ... db999.redb
• DB 0 is reserved for admin metadata
• Each database has its own encryption key
• Values are encrypted with ChaCha20-Poly1305

Building & Publishing

Pre-built Binaries

Pre-built binaries are published to the Forgejo package registry for:

Binary	Platforms	Description
hero_redis	linux-amd64, linux-arm64, darwin-arm64	Redis-compatible server
hero_redis_tester	linux-amd64, linux-arm64, darwin-arm64	Integration test tool
hero_redis_login	linux-amd64, linux-arm64, darwin-arm64	Ed25519 keypair generator & login tool
hero_redis_client	linux-amd64, linux-arm64, darwin-arm64	Client with Rhai scripting

Build Scripts

Two scripts are provided for local development and publishing:

build_package.sh - Local Build & Publish to Forgejo

Builds macOS binaries locally using forgejo-runner and publishes to the Forgejo package registry.

# Set your Forgejo token first
export FORGEJO_TOKEN='your-token-here'

# Run local build and publish
./build_package.sh

This script:

1. Verifies your Forgejo token
2. Installs forgejo-runner if needed
3. Runs the macOS build workflow locally
4. Publishes binaries to forge.ourworld.tf

build_publish.sh - Build & Publish to crates.io

Runs full CI checks and optionally publishes the library to crates.io.

./build_publish.sh

This script:

1. Cleans previous build artifacts
2. Checks code formatting (cargo fmt)
3. Runs clippy linting
4. Builds with release profile
5. Runs tests
6. Generates documentation
7. Verifies package with cargo publish --dry-run
8. Prompts for confirmation before publishing to crates.io

CI/CD Workflows

The Forgejo workflows (.forgejo/workflows/) automatically build and publish on push to main:

• build-linux.yaml: Builds for linux-amd64 and linux-arm64 (with UPX compression)
• build-macos.yaml: Builds for darwin-arm64 (native macOS host)

Both workflows build with --features "server,rhai" to include:

• hero_redis - The server binary
• hero_redis_client - The Rhai-enabled client binary

License

Apache 2.0