fast-jev-compaction asks Jev which tool calls to keep

Claude Code compaction is the moment a long session trades detail for room. When the context window fills, the transcript gets summarized, and tool output you were relying on can leave the conversation: an exact path, a traceback, a command that finally worked.

fast-jev-compaction is a Claude Code plugin that tries to keep raw tool calls and results instead. Before compaction runs, it sends your session state to Jev, a decision model from TypeSafe, and asks two yes-or-no questions per tool call: whether the call still matters, and whether its result should stay. It then keeps or deletes each item against a cutoff, keepThreshold.

Jev is not a chatbot. It is a “System One” model that returns typed probabilities instead of prose, so the plugin can act on the answer directly.

Tested: commit e3f262a7f4d42bd8dd32ced30d26176f7cb545b0 on September 20, 2026. Scope: 120 recorded tool calls scored by Jev, a probe on what the plugin uploads, and a published independent evaluation of 9,471 calls from real sessions. Result: at the shipped keepThreshold 0.5 the plugin kept 0 of 120 calls, the independent evaluation kept 0 of 9,471, and on its behavioral oracle the model’s selection was no better than chance. Verdict: wait.

What the plugin does and what Jev is

It is designed to hook Claude Code’s compaction and replace the built-in summary with its own decision pass, configured to trigger at 60% of the context window.

One setting matters: keepThreshold. Jev scores each item from 0 to 1, and the cutoff decides what stays. The plugin ships with keepThreshold 0.5, and at that setting it keeps nothing.

Jev is TypeSafe’s “System One” model, built to make fast structured decisions rather than write text. It returns typed answers with probabilities, answers many questions about one shared context in parallel, and uses a noul question, a yes-or-no whose answer is the probability that a statement is true. LangChain’s write-up explains the design.

What leaves your machine

We probed what the plugin uploads by planting sentinel strings in each field and searching the request. The split is sharp. Your prompts and the assistant’s text leave verbatim. Tool inputs leave too, including file paths and shell commands, though truncated. Tool results do not: file contents, command output, and error bodies are replaced with a note such as “ok, 51 chars (omitted)”. Jev decides whether to keep a result it has never read.

Jev’s published price is $42 per billion input tokens, with output free. Our 20 requests cost $0.0013 in total, about $0.000066 per request, computed from published pricing. Each request carried a mean of 1,579 input tokens and 220 output tokens, and one four-question probe returned in 367 ms. The community’s cost objection is different: it concerns prompt-cache invalidation upstream, which we did not measure.

The selection claim does not hold

The 120-call corpus cannot settle whether the selection works: its labels were predictable from result length and tool type alone.

An independent evaluation of the same model version is stronger on the one axis that matters. Over about 2,000 real sessions, 91 compression points, and 9,471 tool calls, using a behavioral oracle rather than hand labels, it measured the model’s selection as no better than chance. Between thresholds 0.14 and 0.21 it preserved fewer genuinely-used calls than random selection of the same size. It kept 0 of 9,471 results at the default, matching our 0 of 120. That is an unreviewed community report, but its method is stronger and its default result reproduces ours.

This is a claim about this use of Jev, not about the model generally. What survives: at the shipped 0.5 cutoff the plugin keeps nothing, now confirmed twice; of the 12 surviving messages, 0 had their text changed under a keep decision; and a drop_call decision deletes silently while a drop_result leaves a marker. We corroborated that mechanism behind issue #65 but did not reproduce the nine fabricated turns reported there.

The community record describes a related cycle: each round has less tool bulk left to remove, so compaction frees less each time and eventually falls back to the built-in summary. Two community sources report it, as does issue #70, and our probe shows the same shape on a synthetic transcript.

Only 0.6% of calls, and 3.6% to 7.5% of result tokens, were ever referenced again in that independent corpus. If that holds, the question is less which results to keep than whether keeping any of them justifies a model call per compaction.

The debate and the community record

The community response runs two ways. The main thread and related discussions argue that compaction should be sparing rather than constant, that a scorer which never reads the result cannot judge it (our probe confirms the blindness), that deleting history discards reasoning state and invalidates the prompt cache, that a built-in compactor has an advantage because models are tuned on their own compaction flows, and that injected text can steer the keep decision. The last three are not measured by us.

Positive reports also exist. Some users call the plugin useful without measuring it, and a derivative implementation for a different agent reports verbatim messages surviving compaction and a larger reduction than its stock summarizer. That implementation is not this plugin, and we did not measure it.

The community is wrong on one point. Several posts claim the full contents of tool results reach TypeSafe. Our probe measured the opposite: file contents, command output, and error bodies stay local, while prompts, assistant text, and tool inputs leave.

Wait until the selection is proven

Wait on fast-jev-compaction for long Claude Code sessions, and do not tune the threshold, until its selection is shown to work. The idea is right: replacing a summary with raw tool output that is kept rather than rewritten would be worth a model call. The tested revision is not ready. Its selection signal did not survive independent measurement on real sessions, and we have no threshold we can point you to and say it works.

That is not a verdict on Jev as a model, and it is not a knock on a maintainer whose pinned revision landed the same day the repository was created. We did not exercise plugin registration or hook firing, so we cannot confirm Claude Code loads the plugin or fires the hook. If a later revision shows selection better than chance on a behavioral oracle, documents a key path (issue #54), and we verify the hook fires, check back. Until then, keep Claude Code’s built-in summary compaction as your default.