fix(oschema): delimiter-aware marker parser (closes #67) #78

Merged

timur merged 2 commits from issue-67-marker-parser into development

2026-05-20 05:55:04 +00:00

timur commented

2026-05-20 00:25:40 +00:00

Owner

Summary

Fixes #67. Replaces substring-based [rootobject] / header-delimiter detection in the OSchema parser with a delimiter- and string-literal-aware marker module, and routes the generator through the same helpers so the two crates can't diverge on what counts as a marker.

What changed

New crates/oschema/src/markers.rs — parse_markers, has_marker, strip_markers, is_header_delimiter_line. Tokenises bracketed identifiers, skips OSchema-style "..." literals (with \" escape handling), and rejects malformed brackets ([], [123], unclosed).
oschema/parser.rs — [rootobject] detection now goes through has_marker; the # ===SCHEMA=== split is whole-line via a new split_header helper, so a longer comment that merely mentions the delimiter mid-sentence no longer truncates the schema body.
generator/src/{domain,generator,schemas/oschema}.rs — delegate parse_markers to the oschema crate; remove the remaining c.text.contains("[rootobject]") and s.contains('[') substring checks.

Regression coverage (the failure modes #261 hit)

Adjacent marker collision: [rootobject][index] now yields two distinct markers and the type stays a root object.
Prefix collision: [root] does NOT promote a type to root-object even though [rootobject] would; the two stay independent in both directions.
Markers in string literals: # example: "use [rootobject]" is no longer matched.
Header delimiter whole-line: # Notes — see # ===SCHEMA=== below no longer eats the schema body.

Test plan

11 new unit tests in markers.rs (cargo test -p hero_rpc_oschema --lib markers)
4 new parser-level regression tests in parser.rs (marker_regression_*)
7 new integration tests in oschema/tests/real_schemas.rs parsing every .oschema shipped in the repo (basic OSIS example, recipe_server schemas/ + src/recipes/ + src/recipes/core/, example/recipe_server) — root-object sets unchanged.
79 oschema lib + 7 integration tests pass.
125 generator tests pass.

🤖 Generated with Claude Code

## Summary Fixes #67. Replaces substring-based `[rootobject]` / header-delimiter detection in the OSchema parser with a delimiter- and string-literal-aware marker module, and routes the generator through the same helpers so the two crates can't diverge on what counts as a marker. ## What changed - **New `crates/oschema/src/markers.rs`** — `parse_markers`, `has_marker`, `strip_markers`, `is_header_delimiter_line`. Tokenises bracketed identifiers, skips OSchema-style `"..."` literals (with `\"` escape handling), and rejects malformed brackets (`[]`, `[123]`, unclosed). - **`oschema/parser.rs`** — `[rootobject]` detection now goes through `has_marker`; the `# ===SCHEMA===` split is whole-line via a new `split_header` helper, so a longer comment that merely mentions the delimiter mid-sentence no longer truncates the schema body. - **`generator/src/{domain,generator,schemas/oschema}.rs`** — delegate `parse_markers` to the oschema crate; remove the remaining `c.text.contains("[rootobject]")` and `s.contains('[')` substring checks. ## Regression coverage (the failure modes #261 hit) - **Adjacent marker collision**: `[rootobject][index]` now yields two distinct markers and the type stays a root object. - **Prefix collision**: `[root]` does NOT promote a type to root-object even though `[rootobject]` would; the two stay independent in both directions. - **Markers in string literals**: `# example: "use [rootobject]"` is no longer matched. - **Header delimiter whole-line**: `# Notes — see `# ===SCHEMA===` below` no longer eats the schema body. ## Test plan - [x] 11 new unit tests in `markers.rs` (`cargo test -p hero_rpc_oschema --lib markers`) - [x] 4 new parser-level regression tests in `parser.rs` (`marker_regression_*`) - [x] 7 new integration tests in `oschema/tests/real_schemas.rs` parsing every `.oschema` shipped in the repo (basic OSIS example, recipe_server `schemas/` + `src/recipes/` + `src/recipes/core/`, `example/recipe_server`) — root-object sets unchanged. - [x] 79 oschema lib + 7 integration tests pass. - [x] 125 generator tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

timur added 2 commits

2026-05-20 00:25:40 +00:00

fix(oschema): delimiter-aware marker parser (closes #67 )

Test / test (push) Failing after 2m12s

Details

87a93fbd0f

Replace substring-based `[rootobject]` detection with a dedicated
markers module that tokenises bracketed identifiers, ignores marker-
shaped text inside string literals, and treats adjacent or
prefix-colliding markers as distinct tokens.

- Add crates/oschema/src/markers.rs with parse_markers, has_marker,
  strip_markers, and is_header_delimiter_line. Includes regression
  tests for adjacent markers ([rootobject][index]), prefix collisions
  ([root] vs [rootobject]), string-literal-embedded markers, escaped
  quotes, and whole-line header delimiter matching.
- oschema/parser.rs: route the rootobject detection on type comments
  through has_marker(), and make the # ===SCHEMA=== header split
  whole-line so a longer comment that merely mentions the delimiter
  is no longer treated as the header boundary.

test(oschema): regression tests for marker failure modes + route generator through shared parser (#67 )

Test / test (push) Failing after 2m13s

Details

Test / test (pull_request) Failing after 2m12s

Details

d3324e1420

Adds the regression coverage demanded by the issue and removes the two
remaining substring-style marker checks elsewhere in the workspace so
that the oschema crate is the single source of truth.

oschema/parser.rs
- marker_regression_adjacent_markers: `[rootobject][index]` keeps both
  markers distinct, and the type is still detected as a root object.
- marker_regression_prefix_collision: `[root]` does not promote the
  type to a root object even though `[rootobject]` would; the two
  prefix-colliding marker names stay independent.
- marker_regression_string_literal_mention_does_not_match: a quoted
  example string mentioning `[rootobject]` is no longer parsed as a
  real marker.
- marker_regression_header_delimiter_must_be_whole_line: a comment
  that merely mentions `# ===SCHEMA===` mid-sentence no longer
  truncates the schema body.

oschema/tests/real_schemas.rs
- New integration tests parse every `.oschema` file shipped in the
  repo (basic OSIS example, recipe_server schemas/src/core variants,
  example/recipe_server) and assert the set of root objects is
  unchanged.

generator/domain.rs, generator/schemas/oschema.rs, generator/generator.rs
- Delegate `parse_markers` to the oschema crate; route the remaining
  `c.text.contains("[rootobject]")` and `s.contains('[')` checks
  through `has_marker` / `parse_markers` so the generator and the
  parser cannot diverge on which strings count as markers.