Nearly scaled up a single timed run into an estimate
Office work
- Situation
- Showing a person a rough estimate of how long a task or process will take.
- What the AI did
- It nearly gave an estimate scaled up from a single timed run. Re-measuring first showed times varying about fourfold at the same size, and the estimate was more than three times the actual time.
- How to ask next time (the operator's example)
- "Don't scale an estimate from one run. Measure several times at the same size and several times at other sizes, and tell me how many runs you did. If the spread is large, show a range from the fastest run to the slowest instead of one number."
- One question to ask
- How many runs, at which sizes, will the estimate be based on?
- Check it yourself
- For command-line work,
hyperfineruns a command many times and reports the mean with the min and max. Without it, runtimeat least three times at two sizes and write down every result. - AI used (month)
- Claude Code (2026-09)
- Times this mistake was recorded
- once
- Source
- The operator's work records (the details were changed to a different task)