Pre-download embedder models into Docker image (first-boot delay) #62

New issue

Closed

opened 2026-03-20 03:13:45 +00:00 by mik-tf · 2 comments

mik-tf commented 2026-03-20 03:13:45 +00:00

Owner

Copy link

Problem

hero_osis_server downloads BGE embedding models (~1.4G) from HuggingFace on first boot. This takes 3-5 minutes, during which:

• The OSIS socket doesn't exist yet
• WASM login fails ("Connection error")
• Admin user seeding in OSIS identity is skipped
• All OSIS-dependent features are unavailable

Root cause

build-local.sh builds the ONNX runtime library but does NOT download the embedder models. They're fetched at runtime by hero_embedder_lib::download.

Fix

hero_embedder already has scripts/download-models.sh and make models target. Add a step in build-local.sh (after ONNX runtime download) to:

1. Run download-models.sh to fetch models to dist/var/embedder/models/
2. Update Dockerfile.pack to COPY models into the image
3. OSIS server finds models locally → no download → instant socket creation

Files

• hero_services/docker/build-local.sh — add models download step
• hero_services/Dockerfile.pack — add COPY for models dir
• hero_embedder/scripts/download-models.sh — existing download script

Impact

First-boot time: 3-5 min → ~10s. Login works immediately.

## Problem hero_osis_server downloads BGE embedding models (~1.4G) from HuggingFace on first boot. This takes 3-5 minutes, during which: - The OSIS socket doesn't exist yet - WASM login fails ("Connection error") - Admin user seeding in OSIS identity is skipped - All OSIS-dependent features are unavailable ## Root cause `build-local.sh` builds the ONNX runtime library but does NOT download the embedder models. They're fetched at runtime by `hero_embedder_lib::download`. ## Fix hero_embedder already has `scripts/download-models.sh` and `make models` target. Add a step in `build-local.sh` (after ONNX runtime download) to: 1. Run `download-models.sh` to fetch models to `dist/var/embedder/models/` 2. Update `Dockerfile.pack` to COPY models into the image 3. OSIS server finds models locally → no download → instant socket creation ## Files - `hero_services/docker/build-local.sh` — add models download step - `hero_services/Dockerfile.pack` — add COPY for models dir - `hero_embedder/scripts/download-models.sh` — existing download script ## Impact First-boot time: 3-5 min → ~10s. Login works immediately.

mik-tf referenced this issue 2026-03-20 03:20:24 +00:00

Refactor archipelagos islands from zinit to hero_proc #61

mik-tf commented 2026-03-20 03:20:24 +00:00

Author

Owner

Copy link

Impact assessment (2026-03-20)

OSIS takes ~2 minutes to download embedder models on first boot. This blocks:

1. OSIS identity user seeding — entrypoint runs before socket exists
2. WASM login — gets "Connection error" until OSIS is ready
3. All OSIS-dependent islands — Files, Media, Business, Projects, Communication all blank because OSIS domains arent available
4. hero_osis_seed — cant run seed data until OSIS is ready

Once models are pre-baked in the image, the entrypoint seeding will work, login will be instant, and seed data can populate all domains.

Signed-off-by: mik-tf

## Impact assessment (2026-03-20) OSIS takes ~2 minutes to download embedder models on first boot. This blocks: 1. **OSIS identity user seeding** — entrypoint runs before socket exists 2. **WASM login** — gets "Connection error" until OSIS is ready 3. **All OSIS-dependent islands** — Files, Media, Business, Projects, Communication all blank because OSIS domains arent available 4. **hero_osis_seed** — cant run seed data until OSIS is ready Once models are pre-baked in the image, the entrypoint seeding will work, login will be instant, and seed data can populate all domains. Signed-off-by: mik-tf

mik-tf referenced this issue 2026-03-20 03:25:29 +00:00

Automated browser smoke tests for Hero OS Docker image #64

mik-tf commented 2026-03-20 12:20:25 +00:00

Author

Owner

Copy link

Done

Pre-download BGE embedding models at build time so they are baked into the Docker image. Eliminates the 2-3 min HuggingFace download on first boot.

Changes

• hero_services/docker/build-local.sh: Added step 6/7 to download bge-small, bge-base, and bge-reranker-base models (1.9G) into dist/var/embedder/models/
• hero_services/Dockerfile.pack: COPY pre-downloaded models into image, set EMBEDDER_MODELS env var
• hero_services/docker/entrypoint.sh: Wait up to 240s for OSIS socket before admin seeding (handles case where embedder still loading)
• hero_services/docker/cargo-server-patches.toml: Added herolib_ai and herolib_otoml local path patches

Verification

• Models present in image at /root/hero/var/embedder/models/
• hero_embedder_server starts without downloading — uses pre-baked models
• OSIS admin user seeded correctly after socket wait
• make smoke: hero_embedder_server /health PASS, OSIS user_count > 0 PASS

Commit: hero_services 82cca10

Signed-off-by: mik-tf

## Done Pre-download BGE embedding models at build time so they are baked into the Docker image. Eliminates the 2-3 min HuggingFace download on first boot. ### Changes - **hero_services/docker/build-local.sh**: Added step 6/7 to download bge-small, bge-base, and bge-reranker-base models (1.9G) into `dist/var/embedder/models/` - **hero_services/Dockerfile.pack**: COPY pre-downloaded models into image, set `EMBEDDER_MODELS` env var - **hero_services/docker/entrypoint.sh**: Wait up to 240s for OSIS socket before admin seeding (handles case where embedder still loading) - **hero_services/docker/cargo-server-patches.toml**: Added herolib_ai and herolib_otoml local path patches ### Verification - Models present in image at `/root/hero/var/embedder/models/` - hero_embedder_server starts without downloading — uses pre-baked models - OSIS admin user seeded correctly after socket wait - `make smoke`: hero_embedder_server /health PASS, OSIS user_count > 0 PASS Commit: hero_services 82cca10 Signed-off-by: mik-tf

mik-tf closed this issue 2026-03-20 12:20:25 +00:00

mik-tf referenced this issue 2026-03-20 12:22:43 +00:00

Hero OS — Master Roadmap #38

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

development

main

No results found.

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/home#62

No description provided.