Tested in a shrunk desktop window, not a phone, and nearly called it a bug
Any kind of task
- Situation
- Asking an AI to check whether a sign-up page works on a smartphone.
- What the AI did
- Instead of a phone, it narrowed a desktop browser window and judged the page by a glitch that only happened there. It nearly reported "the button can't be tapped on a phone" for a page that worked.
- How to ask next time (the operator's example)
- "First write one line on how a person would check this, and use that method. If you use a stand-in, say why and when it could give the wrong answer, and add one line to the report on how you checked."
- One question to ask
- In one line: how would a person check this?
- Check it yourself
- Browser device emulation (such as Chrome DevTools' device mode) is a common stand-in. Before trusting a mobile bug report, repeat the step on one real phone, or at least in the iOS Simulator or an Android emulator.
- AI used (month)
- Claude Code (2026-09)
- Times this mistake was recorded
- 5 times
- Source
- The operator's work records (the details were changed to a different task), and videos by Yuru Computer Science Radio (a Japanese YouTube channel)