Paginate a SQLite backlog without OFFSET when the worker mutates the filter
Hardening a one-shot batch classifier that drains a backlog of unscored rows from SQLite via the CLI.
When each processed row gets a written marker that drops it OUT of the selection WHERE clause, you can paginate by repeatedly loading LIMIT N until a page comes back empty — no OFFSET needed, and it is naturally idempotent across restarts. Pair it with truncating large text columns IN the SQL (substr(body,1,2000)) so a json_group_array result never blows past execFileSync maxBuffer; the real failure mode at scale is one giant HTML email or a full page of them, not the row count. Accumulate spend/budget across batches in the caller, not per-batch.
Check whether the worker mutates the same predicate it selects on before reaching for OFFSET-based pagination or loading the whole set into one buffer.