Prompt Adaptation as a Dynamic Complement in Generative AI Systems
Eaman Jahani, Benjamin Manning, Joe Zhang, Hong-Yi TuYe, Mohammed Alsobay, Christos Nicolaides, Siddharth Suri, and David Holtz
Information Systems Research 2026
Do users automatically benefit when generative AI models improve, or does realizing those gains require changing how they interact with the technology? We ran two preregistered experiments with 3,750 participants submitting nearly 37,000 prompts across two versions of OpenAI’s DALL-E to find out. The answer depends critically on task structure. In a bounded task with a clear objective, replicating a specific target image, roughly half of the performance gain came from the model itself, and half came from users naturally adjusting their prompts. In an open-ended creative task, designing logos for hypothetical organizations, nearly all of the improvement was attributable to the model alone. We then asked whether automated prompt rewriting, a feature embedded in many commercial AI products, could substitute for this user adaptation. It could not: rewriting modestly helped in the creative task but substantially hurt performance in the replication task, where precise user control mattered most. For organizations deploying AI in precision-oriented workflows, our results point to an underappreciated priority: investing in how people learn to interact with a model may matter as much as the model itself.