lhumina_code/hero_lib

Fork 1

support gemini-3.1-flash-image-preview #121

New issue

Open

opened 2026-03-20 06:27:16 +00:00 by despiegk · 4 comments

despiegk commented

2026-03-20 06:27:16 +00:00

Owner

is https://openrouter.ai/google/gemini-3.1-flash-image-preview supported in the ai client
if not we need to implement it

see crates/ai

check the specific parameters needed
maybe make specific module for it

we need to be able to specify quality, size...

check https://openrouter.ai/docs/guides/overview/multimodal/image-generation#image-aspect-ratio-configuration

is https://openrouter.ai/google/gemini-3.1-flash-image-preview supported in the ai client if not we need to implement it see crates/ai check the specific parameters needed maybe make specific module for it we need to be able to specify quality, size... check https://openrouter.ai/docs/guides/overview/multimodal/image-generation#image-aspect-ratio-configuration

despiegk commented

2026-03-20 06:31:42 +00:00

Author

Owner

Implementation Spec for Issue #121: Support gemini-3.1-flash-image-preview

Objective

Add support for the google/gemini-3.1-flash-image-preview model in the herolib_ai crate, including a new image generation module that handles OpenRouter's image generation API via the /chat/completions endpoint with modalities and image_config parameters. Usable through both Rust API and Rhai scripting bindings.

Requirements

Add Gemini3_1FlashImagePreview as a new variant in the Model enum
Create a new image_generation module with types for image generation requests, options, and responses
Extend ChatCompletionRequest to support modalities and image_config fields
Extend ResponseMessage to support images array in responses
Add image generation methods to AiClient
Add Rhai bindings for image generation functions
Document all new public API surface
No C-based dependencies

Files to Modify/Create

File	Action	Description
`crates/ai/src/model.rs`	Modify	Add Gemini model variant with OpenRouter provider mapping
`crates/ai/src/image_generation/mod.rs`	Create	New module: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse
`crates/ai/src/types.rs`	Modify	Add modalities, image_config to request; images to response
`crates/ai/src/client/mod.rs`	Modify	Add generate_image methods to AiClient
`crates/ai/src/lib.rs`	Modify	Add module declaration and re-exports
`crates/ai/src/rhai.rs`	Modify	Add Rhai bindings for image generation
`crates/ai/Cargo.toml`	Modify	Add base64 dependency if needed

Implementation Plan

Step 1: Add Gemini model variant to model.rs

Add Gemini3_1FlashImagePreview to Model enum
Add all match arms (info, name, id, from_id, all)
Add Model::default_image_generation() method

Step 2: Create image_generation module

ImageAspectRatio enum (1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1)
ImageSize enum (0.5K, 1K, 2K, 4K)
ImageConfig builder struct
ImageGenerationResponse struct

Step 3: Extend types.rs with image request/response fields

Add modalities and image_config to ChatCompletionRequest
Add ImageConfigRequest struct
Add ImageContent/ImageUrl to response types

Step 4: Add image generation methods to AiClient

generate_image(prompt) - simple default
generate_image_with_options(model, prompt, config) - full control
generate_image_to_file(model, prompt, path, config) - save to file
Base64 decoding of response images

Step 5: Update lib.rs with module declaration and re-exports

Step 6: Add Rhai bindings

ai_generate_image(prompt, output_path)
ai_generate_image_with_options(prompt, aspect_ratio, image_size, output_path)
ai_image_aspect_ratios() and ai_image_sizes()

Step 7: Add examples and documentation

Acceptance Criteria

Model::Gemini3_1FlashImagePreview is valid and maps to OpenRouter
ImageConfig builder works with aspect ratio and image size
ChatCompletionRequest serializes modalities and image_config correctly
ResponseMessage deserializes image responses
AiClient::generate_image() sends correctly structured requests
Rhai bindings are registered and callable
cargo test -p herolib_ai passes
cargo doc -p herolib_ai builds cleanly
No C-based dependencies added

Notes

Image generation uses the chat completions endpoint with modalities: ["image", "text"]
Extended aspect ratios (1:4, 4:1, 1:8, 8:1) are Gemini 3.1 Flash specific
Response images come as data:image/png;base64,... URLs that need decoding

🤖 Generated with Claude Code

## Implementation Spec for Issue #121: Support gemini-3.1-flash-image-preview ### Objective Add support for the `google/gemini-3.1-flash-image-preview` model in the `herolib_ai` crate, including a new image generation module that handles OpenRouter's image generation API via the `/chat/completions` endpoint with `modalities` and `image_config` parameters. Usable through both Rust API and Rhai scripting bindings. ### Requirements - Add `Gemini3_1FlashImagePreview` as a new variant in the `Model` enum - Create a new `image_generation` module with types for image generation requests, options, and responses - Extend `ChatCompletionRequest` to support `modalities` and `image_config` fields - Extend `ResponseMessage` to support `images` array in responses - Add image generation methods to `AiClient` - Add Rhai bindings for image generation functions - Document all new public API surface - No C-based dependencies ### Files to Modify/Create | File | Action | Description | |------|--------|-------------| | `crates/ai/src/model.rs` | Modify | Add Gemini model variant with OpenRouter provider mapping | | `crates/ai/src/image_generation/mod.rs` | Create | New module: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse | | `crates/ai/src/types.rs` | Modify | Add modalities, image_config to request; images to response | | `crates/ai/src/client/mod.rs` | Modify | Add generate_image methods to AiClient | | `crates/ai/src/lib.rs` | Modify | Add module declaration and re-exports | | `crates/ai/src/rhai.rs` | Modify | Add Rhai bindings for image generation | | `crates/ai/Cargo.toml` | Modify | Add base64 dependency if needed | ### Implementation Plan #### Step 1: Add Gemini model variant to model.rs - Add `Gemini3_1FlashImagePreview` to `Model` enum - Add all match arms (info, name, id, from_id, all) - Add `Model::default_image_generation()` method #### Step 2: Create image_generation module - `ImageAspectRatio` enum (1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1) - `ImageSize` enum (0.5K, 1K, 2K, 4K) - `ImageConfig` builder struct - `ImageGenerationResponse` struct #### Step 3: Extend types.rs with image request/response fields - Add `modalities` and `image_config` to `ChatCompletionRequest` - Add `ImageConfigRequest` struct - Add `ImageContent`/`ImageUrl` to response types #### Step 4: Add image generation methods to AiClient - `generate_image(prompt)` - simple default - `generate_image_with_options(model, prompt, config)` - full control - `generate_image_to_file(model, prompt, path, config)` - save to file - Base64 decoding of response images #### Step 5: Update lib.rs with module declaration and re-exports #### Step 6: Add Rhai bindings - `ai_generate_image(prompt, output_path)` - `ai_generate_image_with_options(prompt, aspect_ratio, image_size, output_path)` - `ai_image_aspect_ratios()` and `ai_image_sizes()` #### Step 7: Add examples and documentation ### Acceptance Criteria - [ ] `Model::Gemini3_1FlashImagePreview` is valid and maps to OpenRouter - [ ] `ImageConfig` builder works with aspect ratio and image size - [ ] `ChatCompletionRequest` serializes modalities and image_config correctly - [ ] `ResponseMessage` deserializes image responses - [ ] `AiClient::generate_image()` sends correctly structured requests - [ ] Rhai bindings are registered and callable - [ ] `cargo test -p herolib_ai` passes - [ ] `cargo doc -p herolib_ai` builds cleanly - [ ] No C-based dependencies added ### Notes - Image generation uses the chat completions endpoint with `modalities: ["image", "text"]` - Extended aspect ratios (1:4, 4:1, 1:8, 8:1) are Gemini 3.1 Flash specific - Response images come as `data:image/png;base64,...` URLs that need decoding --- 🤖 Generated with [Claude Code](https://claude.com/claude-code)

despiegk commented

2026-03-20 06:34:46 +00:00

Author

Owner

Updated Implementation Spec for Issue #121: Support gemini-3.1-flash-image-preview

Objective

Requirements

Add Gemini3_1FlashImagePreview as a new variant in the Model enum
Create a new image_generation module with types for image generation requests, options, and responses
Extend ChatCompletionRequest to support modalities and image_config fields
Extend ResponseMessage to support images array in responses
Add image generation methods to AiClient
Write the resulting image file to a user-defined output path
Add Rhai bindings for image generation functions
Write Rust example code in examples/
Test with the hero_browser MCP tool to visually verify generated images
Proper unit tests and integration tests
Document all new public API surface
No C-based dependencies

Files to Modify/Create

File	Action	Description
`crates/ai/src/model.rs`	Modify	Add Gemini model variant with OpenRouter provider mapping
`crates/ai/src/image_generation/mod.rs`	Create	New module: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse
`crates/ai/src/types.rs`	Modify	Add modalities, image_config to request; images to response
`crates/ai/src/client/mod.rs`	Modify	Add generate_image methods to AiClient
`crates/ai/src/lib.rs`	Modify	Add module declaration and re-exports
`crates/ai/src/rhai.rs`	Modify	Add Rhai bindings for image generation
`crates/ai/Cargo.toml`	Modify	Add base64 dependency if needed
`crates/ai/examples/image_generation.rs`	Create	Rust example demonstrating image generation with various options, writing output to file

Implementation Plan

Step 1: Add Gemini model variant to model.rs

Add Gemini3_1FlashImagePreview to Model enum
Add all match arms (info, name, id, from_id, all)
Add Model::default_image_generation() method

Step 2: Create image_generation module

ImageAspectRatio enum (1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1)
ImageSize enum (0.5K, 1K, 2K, 4K)
ImageConfig builder struct
ImageGenerationResponse struct

Step 3: Extend types.rs with image request/response fields

Add modalities and image_config to ChatCompletionRequest
Add ImageConfigRequest struct
Add ImageContent/ImageUrl to response types

Step 4: Add image generation methods to AiClient

generate_image(prompt) - simple default
generate_image_with_options(model, prompt, config) - full control
generate_image_to_file(model, prompt, path, config) - save resulting image to a user-defined file path
Base64 decoding of response images

Step 5: Update lib.rs with module declaration and re-exports

Step 6: Add Rhai bindings

ai_generate_image(prompt, output_path)
ai_generate_image_with_options(prompt, aspect_ratio, image_size, output_path)
ai_image_aspect_ratios() and ai_image_sizes()

Step 7: Write Rust example code

Create crates/ai/examples/image_generation.rs with working example
Demonstrate generating images with different aspect ratios and sizes
Write output image to a defined file path

Step 8: Testing

Unit tests for enum parsing, config builder, serialization/deserialization
Integration test that actually calls the API to generate an image
Use hero_browser MCP to open and visually verify the generated image file
Verify cargo test -p herolib_ai passes

Acceptance Criteria

Model::Gemini3_1FlashImagePreview is valid and maps to OpenRouter
ImageConfig builder works with aspect ratio and image size
ChatCompletionRequest serializes modalities and image_config correctly
ResponseMessage deserializes image responses
AiClient::generate_image() sends correctly structured requests
Generated image is written to the user-specified output file path
Rust example in examples/ compiles and runs correctly
Rhai bindings are registered and callable
Unit tests pass for all new types and serialization
Integration test generates a real image via OpenRouter API
Generated image visually verified via hero_browser MCP
cargo doc -p herolib_ai builds cleanly
No C-based dependencies added

Notes

Image generation uses the chat completions endpoint with modalities: ["image", "text"]
Extended aspect ratios (1:4, 4:1, 1:8, 8:1) are Gemini 3.1 Flash specific
Response images come as data:image/png;base64,... URLs that need decoding
Testing with hero_browser MCP allows visual verification that generated images are valid

🤖 Generated with Claude Code

## Updated Implementation Spec for Issue #121: Support gemini-3.1-flash-image-preview ### Objective Add support for the `google/gemini-3.1-flash-image-preview` model in the `herolib_ai` crate, including a new image generation module that handles OpenRouter's image generation API via the `/chat/completions` endpoint with `modalities` and `image_config` parameters. Usable through both Rust API and Rhai scripting bindings. ### Requirements - Add `Gemini3_1FlashImagePreview` as a new variant in the `Model` enum - Create a new `image_generation` module with types for image generation requests, options, and responses - Extend `ChatCompletionRequest` to support `modalities` and `image_config` fields - Extend `ResponseMessage` to support `images` array in responses - Add image generation methods to `AiClient` - **Write the resulting image file to a user-defined output path** - Add Rhai bindings for image generation functions - **Write Rust example code in `examples/`** - **Test with the hero_browser MCP tool to visually verify generated images** - **Proper unit tests and integration tests** - Document all new public API surface - No C-based dependencies ### Files to Modify/Create | File | Action | Description | |------|--------|-------------| | `crates/ai/src/model.rs` | Modify | Add Gemini model variant with OpenRouter provider mapping | | `crates/ai/src/image_generation/mod.rs` | Create | New module: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse | | `crates/ai/src/types.rs` | Modify | Add modalities, image_config to request; images to response | | `crates/ai/src/client/mod.rs` | Modify | Add generate_image methods to AiClient | | `crates/ai/src/lib.rs` | Modify | Add module declaration and re-exports | | `crates/ai/src/rhai.rs` | Modify | Add Rhai bindings for image generation | | `crates/ai/Cargo.toml` | Modify | Add base64 dependency if needed | | `crates/ai/examples/image_generation.rs` | Create | **Rust example demonstrating image generation with various options, writing output to file** | ### Implementation Plan #### Step 1: Add Gemini model variant to model.rs - Add `Gemini3_1FlashImagePreview` to `Model` enum - Add all match arms (info, name, id, from_id, all) - Add `Model::default_image_generation()` method #### Step 2: Create image_generation module - `ImageAspectRatio` enum (1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1) - `ImageSize` enum (0.5K, 1K, 2K, 4K) - `ImageConfig` builder struct - `ImageGenerationResponse` struct #### Step 3: Extend types.rs with image request/response fields - Add `modalities` and `image_config` to `ChatCompletionRequest` - Add `ImageConfigRequest` struct - Add `ImageContent`/`ImageUrl` to response types #### Step 4: Add image generation methods to AiClient - `generate_image(prompt)` - simple default - `generate_image_with_options(model, prompt, config)` - full control - **`generate_image_to_file(model, prompt, path, config)` - save resulting image to a user-defined file path** - Base64 decoding of response images #### Step 5: Update lib.rs with module declaration and re-exports #### Step 6: Add Rhai bindings - `ai_generate_image(prompt, output_path)` - `ai_generate_image_with_options(prompt, aspect_ratio, image_size, output_path)` - `ai_image_aspect_ratios()` and `ai_image_sizes()` #### Step 7: Write Rust example code - **Create `crates/ai/examples/image_generation.rs`** with working example - Demonstrate generating images with different aspect ratios and sizes - Write output image to a defined file path #### Step 8: Testing - **Unit tests** for enum parsing, config builder, serialization/deserialization - **Integration test** that actually calls the API to generate an image - **Use hero_browser MCP to open and visually verify the generated image file** - Verify `cargo test -p herolib_ai` passes ### Acceptance Criteria - [ ] `Model::Gemini3_1FlashImagePreview` is valid and maps to OpenRouter - [ ] `ImageConfig` builder works with aspect ratio and image size - [ ] `ChatCompletionRequest` serializes modalities and image_config correctly - [ ] `ResponseMessage` deserializes image responses - [ ] `AiClient::generate_image()` sends correctly structured requests - [ ] **Generated image is written to the user-specified output file path** - [ ] **Rust example in `examples/` compiles and runs correctly** - [ ] Rhai bindings are registered and callable - [ ] **Unit tests pass for all new types and serialization** - [ ] **Integration test generates a real image via OpenRouter API** - [ ] **Generated image visually verified via hero_browser MCP** - [ ] `cargo doc -p herolib_ai` builds cleanly - [ ] No C-based dependencies added ### Notes - Image generation uses the chat completions endpoint with `modalities: ["image", "text"]` - Extended aspect ratios (1:4, 4:1, 1:8, 8:1) are Gemini 3.1 Flash specific - Response images come as `data:image/png;base64,...` URLs that need decoding - Testing with hero_browser MCP allows visual verification that generated images are valid --- 🤖 Generated with [Claude Code](https://claude.com/claude-code)

despiegk commented

2026-03-20 06:46:34 +00:00

Author

Owner

Test Results

Unit Tests

Total: 52 (49 lib + 3 integration)
Passed: 52
Failed: 0

cargo test -p herolib_ai --features rhai
test result: ok. 49 passed; 0 failed; 0 ignored

Integration Tests (Live API)

Test 1: Square image (1:1, 1K) — OK (491,500 bytes)
Test 2: Widescreen image (16:9, 1K) — OK (1,899,154 bytes)
Test 3: Generate with response metadata — OK (1,375,445 bytes)

All images visually verified via hero_browser MCP:

Square: Red circle on white background ✓
Widescreen: Mountain landscape at sunset ✓
Default: Cartoon cat on rainbow ✓

Files Changed

crates/ai/src/model.rs — Added Gemini3_1FlashImagePreview model variant
crates/ai/src/image_generation/mod.rs — NEW: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse
crates/ai/src/types.rs — Added modalities, image_config, images fields + null content handling
crates/ai/src/client/mod.rs — Added generate_image methods, base64 decoder
crates/ai/src/lib.rs — Module declaration and re-exports
crates/ai/src/rhai.rs — Rhai bindings for image generation
crates/ai/src/bin/image_gen_test.rs — Integration test binary
crates/ai/examples/rust/image_generation_test.rs — Rust example
crates/ai/examples/rhai/05_image_generation.rhai — Rhai example

🤖 Generated with Claude Code

## Test Results ### Unit Tests - **Total: 52** (49 lib + 3 integration) - **Passed: 52** - **Failed: 0** ``` cargo test -p herolib_ai --features rhai test result: ok. 49 passed; 0 failed; 0 ignored ``` ### Integration Tests (Live API) - **Test 1**: Square image (1:1, 1K) — OK (491,500 bytes) - **Test 2**: Widescreen image (16:9, 1K) — OK (1,899,154 bytes) - **Test 3**: Generate with response metadata — OK (1,375,445 bytes) All images visually verified via hero_browser MCP: - Square: Red circle on white background ✓ - Widescreen: Mountain landscape at sunset ✓ - Default: Cartoon cat on rainbow ✓ ### Files Changed - `crates/ai/src/model.rs` — Added Gemini3_1FlashImagePreview model variant - `crates/ai/src/image_generation/mod.rs` — NEW: ImageConfig, ImageAspectRatio, ImageSize, ImageGenerationResponse - `crates/ai/src/types.rs` — Added modalities, image_config, images fields + null content handling - `crates/ai/src/client/mod.rs` — Added generate_image methods, base64 decoder - `crates/ai/src/lib.rs` — Module declaration and re-exports - `crates/ai/src/rhai.rs` — Rhai bindings for image generation - `crates/ai/src/bin/image_gen_test.rs` — Integration test binary - `crates/ai/examples/rust/image_generation_test.rs` — Rust example - `crates/ai/examples/rhai/05_image_generation.rhai` — Rhai example 🤖 Generated with [Claude Code](https://claude.com/claude-code)