Skip to main content
  1. Today I Learned (TIL)/

Fix the spec, not the prompt

·208 words·1 min·

Today I spent twenty minutes re-wording a request to an AI before I realized the wording was never the problem. I was building a cleanup rule for my Calibre library (collapse duplicate books) and the model kept getting it subtly wrong. I kept rephrasing, trying a cleverer prompt. Nothing.

Then it landed: I’d never actually defined what “duplicate” meant. Same title? Same title and author? Same author, different edition, is that one book or two? I had a fuzzy idea in my head and was annoyed the model couldn’t read it. It wasn’t misunderstanding my prompt. It was faithfully executing an underspecified one.

The moment I wrote the rule down, the model got it on the first try: match on normalized title plus author, treat differing formats as one book, flag everything else for review instead of merging. The fix wasn’t a better prompt. It was a spec.

Prompting is just specification, and engineers have always been a bit bad at it: we carry the requirements in our heads and resent that they aren’t obvious to everyone else. When the model keeps missing, the tell isn’t “phrase it differently.” It’s “I haven’t decided what I actually want yet.” Vague in, vague out, no matter how nicely you ask.

Chandler Thompson
Author
Chandler Thompson
I lead engineering teams and coach the people who run them. This is where I write down what actually worked.

Related