The Labs Have the Best Models, So Why Do Their AI Apps Still Kind of Suck?

The rapid advancement of artificial intelligence has led to the creation of powerful "frontier models" capable of sophisticated tasks, including code generation. However, a prominent question circulating within the tech community, particularly among Apple enthusiasts, is why many AI applications developed by major labs continue to offer a subpar user experience. Ben Davis, co-host of the podcast Nerd Snipe, has articulated a compelling perspective on this phenomenon, suggesting that the issue transcends the technical capabilities of AI models and delves into the realm of human discernment and aesthetic judgment.

The "Kitchen Sink" Syndrome in AI Application Design

Davis’s analysis, shared in a recent publication, was prompted by observations of applications like the ChatGPT/Codex desktop app. He characterizes such software as succumbing to a "kitchen-sink design syndrome," where an overwhelming array of features are crammed into user interfaces that become cluttered and unintuitive. This often manifests as overly complex navigation, the inclusion of multiple, often redundant, sidebars, and a confusing arrangement of menus and functionalities.

The core of the paradox, as Davis points out, is the apparent disconnect between the advanced capabilities of the underlying AI models and the perceived decline in product quality. "To me, the most interesting thing here is the implied question of ‘if the Codex team has unlimited tokens on really powerful unreleased models, why is the product getting worse?’" Davis posited. He explicitly dismisses the notion of "vibe coding" – a colloquial term implying a lack of rigorous development – as the primary culprit, labeling those who cling to this explanation as misinformed. "Anyone who still says this is a moron and you can ignore them," he stated unequivocally.

Davis argues that modern AI models, particularly those capable of generating code, are remarkably proficient. "Frankly, these models are better at writing code than pretty much anyone at this point," he asserts. "You tell the model to make something and it’ll make that thing quite well." He further contextualizes this by noting that the inherent complexity and user-unfriendliness of some software, such as Jira, predates the widespread adoption of large language models (LLMs). Davis believes that when utilized effectively, AI agents can actually lead to the creation of significantly improved products, citing examples like T3 Code and Grok Bot as evidence of this potential.

The Unquantifiable Elements of Good Design

This debate is not entirely new. In the realm of software development, the evolution of application quality has been a consistent theme. A seasoned app reviewer with over 17 years of experience noted that even the most rudimentary applications built with advanced AI models like Anthropic’s Opus 5.5 today exhibit a stability and user-friendliness far exceeding that of early applications developed by amateur human programmers in 2009. This observation reinforces Davis’s central argument: if the technical underpinnings are so robust, why do so many AI-powered applications still fall short?

The answer, according to discussions involving AI experts, including Andrew Ambrosino of OpenAI, points to the limitations of current AI in replicating uniquely human qualities. These include human discernment, taste, and an intuitive understanding of user experience – attributes that are notoriously difficult to quantify and translate into metrics that AI models can reliably process. While AI excels at optimization tasks with clear parameters, it struggles with subjective decision-making.

Taste as a Differentiator in AI Development

Davis elaborates on this concept by defining "taste" in the context of software development as "the ability to make good decisions in places where there isn’t a programmatic verification function." He differentiates this from purely technical tasks. For instance, optimizing code for speed ("make this function really fast") is a quantifiable task that AI models can excel at through iterative testing and refinement, akin to a "hill climbing" process. Davis suggests that models like Opus 5.5 can effectively solve any problem expressible in this format.

However, subjective aesthetic and usability choices, such as selecting preferred color palettes or designing intuitive dashboard layouts, fall into the domain of "taste." These decisions cannot be objectively verified by an algorithm; they rely on human preference and judgment. "It’s a simple little example, but everything from codebase structure to dashboard layout to how features in ChatGPT fit together all fit under here," Davis explains. He concludes that while AI-generated implementations may be technically "correct" and pass all tests, the underlying choices made by the AI or the developers guiding it can still result in a product that is aesthetically displeasing or functionally awkward to the human user.

The Human Element in an AI-Driven World

This sentiment resonates strongly with those who interact with AI coding agents on a daily basis. The phrase, "The tests pass, I just hate the choices," encapsulates a common frustration. For instance, in the development of new applications for MacStories, the process involves extensive consideration of usability, onboarding flows, platform best practices, and subjective "feel" – aspects that current AI models have not yet been able to master. The rush by major labs to maximize token usage and build "everything because why not?" may have inadvertently led to an overlooking of these critical, qualitative aspects of software design. The implication is that a truly great application experience involves more than just elegant, test-passing code.

The pace of innovation in the AI sector is extraordinary, and perspectives are likely to evolve rapidly. However, at the current juncture, Davis’s analysis provides a crucial framework for understanding the persistent gap between the technical prowess of AI models and the user experience of AI-powered applications. The continued reliance on human judgment, taste, and discernment remains a significant factor in shaping the quality of software, even when developed with the assistance of advanced AI agents.

Supporting Data and Context

The development of sophisticated AI models capable of complex tasks such as code generation has accelerated significantly over the past few years. Leading AI research labs, including OpenAI, Google DeepMind, and Anthropic, have invested billions of dollars in developing LLMs like GPT-4, Gemini, and Claude. These models have demonstrated remarkable capabilities in understanding natural language, generating creative text formats, and, crucially, writing and debugging code.

For example, OpenAI’s Codex, a descendant of GPT-3, was specifically trained on billions of lines of publicly available code from sources like GitHub. This training enabled it to translate natural language prompts into code across various programming languages. Similarly, Anthropic’s Claude models, particularly Opus 5.5, have been lauded for their advanced reasoning and coding abilities.

Despite these advancements, the user-facing applications built upon these models have often faced criticism. Early versions of ChatGPT’s desktop application, for instance, were noted for their complexity. Similarly, numerous third-party AI coding assistants, while offering powerful backend capabilities, have struggled with intuitive user interfaces and seamless integration into existing development workflows.

Chronology of AI Development and Application Design

• Early 2010s: The rise of mobile computing and app stores led to a focus on streamlined user interfaces and platform-specific design principles.
• Mid-2010s: Advancements in machine learning began to influence application development, with AI features becoming more prevalent, though often backend-focused.
• Late 2010s – Early 2020s: The emergence of powerful LLMs like GPT-3 and subsequent iterations marked a significant leap in AI’s ability to understand and generate human-like text and code. This period saw a surge in AI-powered tools and applications.
• 2022-2023: Public release and widespread adoption of generative AI models, leading to a boom in AI application development. Initial user experiences with some of these applications highlighted design challenges.
• 2024: Ongoing discourse and analysis, such as Ben Davis’s take, focusing on the discrepancy between AI model capabilities and application usability, emphasizing the role of human judgment and taste.

Broader Implications and Future Outlook

The insights from Davis and other industry observers have significant implications for the future of software development and AI product design. As AI becomes more integrated into creative and technical workflows, the distinction between AI-generated output and human-curated experience will become increasingly important.

For AI labs and developers, this underscores the necessity of investing not only in the core AI models but also in user experience (UX) and user interface (UI) design. A deep understanding of human psychology, aesthetic principles, and the nuances of user interaction will be crucial for creating AI applications that are not just functional but also delightful to use.

The development of AI agents that can effectively collaborate with humans, rather than simply automate tasks, will likely hinge on their ability to grasp subjective preferences and contextual understanding. Future iterations of AI tools may need to incorporate more sophisticated mechanisms for learning and adapting to individual user tastes and workflows.

Furthermore, the increasing complexity of AI models and their applications may necessitate new approaches to development, potentially involving a more hybrid model where human designers and developers work in tandem with AI agents, each contributing their unique strengths. The ultimate goal remains the creation of powerful, intuitive, and aesthetically pleasing software that enhances human productivity and creativity. The current challenges, while significant, represent a transitional phase in the ongoing evolution of human-AI collaboration.