Back to explore

ANN Dedup Experiment with Jev: Quality Up, Costly

Jonathan Ellis shares his ANN-powered deduplication experiment where Jev classifies top-k candidates. Pros: ~10k lines of code removed and 30 subtle bugs fixed caused by near-duplicates. Cons: not cheap or fast; ~$166 API cost for 207M Opus tokens and nearly 8B Luna subagent tokens (via mjolnir).

Results of my ANN-powered deduplication experiment, with top-k candidates classified by Jev. Pro: -10kloc, fixed 30 bugs where one near-duplicate was subtly broken Con: not cheap and not fast. $166 at API pricing for 207M Opus tokens and nearly 8B in Luna subagents (via mjolnir

ANN Dedup Experiment with Jev: Quality Up, Costly 1
· 0 likesOpen on X