UI: Add Documents button does not commit — added docs invisible until manual index.commit #16

New issue

Closed

opened 2026-04-27 12:03:10 +00:00 by rawan · 0 comments

rawan commented

2026-04-27 12:03:10 +00:00

Member

Summary

In the hero_indexer admin UI, clicking Add Documents appears to succeed (success alert, opstamp returned), but the documents are not actually visible afterwards: doc count stays the same, queries return the old set, and db.info / schema.get reflect no growth.

Root cause

The addDocuments() handler in crates/hero_indexer_ui/static/index.html calls doc.add_batch and stops there:

await rpcCall("db.select", { name: db });
const result = await rpcCall("doc.add_batch", { documents: docs });
// no commit, no reload

Tantivy holds added documents in the writer's in-memory queue until commit() is called, and the reader does not see them until reload() is called against a committed segment. So the RPC doc.add_batch correctly returns an opstamp, but readers (stats, queries, list) keep showing the old state.

The Run Performance Test path in the same file does it correctly:

await rpcCall("doc.add_batch", { documents: docs });
// ...
await rpcCall("index.commit");
await rpcCall("index.reload");

Reproduction

Start hero_indexer + UI.
Select a database.
Paste a few JSON docs into the Add Documents textarea, click Add Documents.
Observe the green success alert.
Refresh stats / run a query — count is unchanged, new docs are absent.
Manually call index.commit then index.reload via the API explorer — docs now appear.

Proposed fix

After a successful doc.add_batch, the addDocuments() UI handler should call index.commit followed by index.reload, the same way the perf-test flow already does. Alternative: add an auto_commit option to doc.add / doc.add_batch on the server so the RPC layer always leaves the index in a queryable state for interactive callers.

Notes

The success message saying "Successfully added N document(s)" is misleading without the commit — at minimum the message should reflect that the docs are committed and visible.
Same gap likely exists for any other UI flow that adds/deletes documents without committing — worth auditing doc.delete and any future single-doc add.

## Summary In the hero_indexer admin UI, clicking **Add Documents** appears to succeed (success alert, opstamp returned), but the documents are not actually visible afterwards: doc count stays the same, queries return the old set, and `db.info` / `schema.get` reflect no growth. ## Root cause The `addDocuments()` handler in [crates/hero_indexer_ui/static/index.html](crates/hero_indexer_ui/static/index.html) calls `doc.add_batch` and stops there: ```js await rpcCall("db.select", { name: db }); const result = await rpcCall("doc.add_batch", { documents: docs }); // no commit, no reload ``` Tantivy holds added documents in the writer's in-memory queue until `commit()` is called, and the reader does not see them until `reload()` is called against a committed segment. So the RPC `doc.add_batch` correctly returns an opstamp, but readers (stats, queries, list) keep showing the old state. The **Run Performance Test** path in the same file does it correctly: ```js await rpcCall("doc.add_batch", { documents: docs }); // ... await rpcCall("index.commit"); await rpcCall("index.reload"); ``` ## Reproduction 1. Start hero_indexer + UI. 2. Select a database. 3. Paste a few JSON docs into the Add Documents textarea, click **Add Documents**. 4. Observe the green success alert. 5. Refresh stats / run a query — count is unchanged, new docs are absent. 6. Manually call `index.commit` then `index.reload` via the API explorer — docs now appear. ## Proposed fix After a successful `doc.add_batch`, the `addDocuments()` UI handler should call `index.commit` followed by `index.reload`, the same way the perf-test flow already does. Alternative: add an `auto_commit` option to `doc.add` / `doc.add_batch` on the server so the RPC layer always leaves the index in a queryable state for interactive callers. ## Notes - The success message saying "Successfully added N document(s)" is misleading without the commit — at minimum the message should reflect that the docs are committed and visible. - Same gap likely exists for any other UI flow that adds/deletes documents without committing — worth auditing `doc.delete` and any future single-doc add.