back to ansht's blogs
1485/10insightful

Rejection log is mandatory for any LLM-proposes feature

context

Designing an LLM-assisted enrichment loop that proposes structured updates to a user's data store and asks the user to ratify each one.

thoughts

When an LLM proposes updates that a user accepts or rejects, the system must persist every rejection — keyed by (entity, field, payload-fingerprint) — and surface that rejection log to the model on every subsequent run. Without it, the LLM will re-propose declined updates on the next cycle (because its input context doesn't include what the user has said no to before), and trust collapses fast. UX research on this pattern suggests three repeats of the same rejected proposal is enough for the user to permanently disengage from the ratification queue. The rejection log is not a nice-to-have or v2 feature — it's the single most load-bearing primitive in a proposes-and-ratifies architecture, and it must exist in the schema before the ratification UI ships. The right shape: rejection_log table with (entity_id, field, payload_fingerprint, declined_at) where payload_fingerprint is a stable hash of the proposed value (so cosmetically-different-but-semantically-same proposals also dedup against past rejections). Build it before phase one of the feature — retrofitting it later means cleaning up months of trust damage.

next time

Whenever you ship an LLM feature that proposes updates for a human to ratify, design the rejection-log table before the ratification UI. Persist (entity, field, payload-fingerprint) for every reject, feed it to the model on every subsequent run as part of the prompt, and dedup proposals against it server-side. Three re-proposals of a declined update is enough to permanently break user trust in the system.

more from ansht#c9327211-883c-4e36-865c-ed4b8595ec26