A single sentence at a Stanford GSB fireside chat fundamentally changed how I think about building AI agents. After weeks of interviewing practitioners and assuming that model intelligence was the primary driver of agent success, the co-founder of a leading autonomous-agent startup said something that stopped me cold: "Model only contributes 30-40% of the whole thing." This was not speculation from an outsider - it was the hard-won lesson from 10 months of building one of the most successful AI agent products in the market. In that moment, everything I thought I knew about what makes agents work inverted.
The Discovery
I walked into the startup's fireside chat in September 2025 with the mental model that most people in AI share: better models lead to better agents. The industry narrative reinforced this constantly - every new GPT version, every Claude update, every Gemini release promised more capable agents. Valuations followed frontier model announcements. The path forward seemed clear: wait for smarter models, build agents on top.
The startup's team had been building their "action engine" for 10 months by that point. Unlike ChatGPT or Perplexity, which they positioned as "answer engines," their product was designed to actually get things done - not just provide information, but execute multi-step tasks end-to-end. They had shipped product, acquired users globally across markets from the US to Japan to Brazil, and learned hard lessons about what actually determines whether an agent succeeds or fails.
What they discovered contradicted the industry's core assumption. They started with the same belief everyone else had - that models would contribute 70% or more to agent success. After 10 months in production with real users and continuous iteration, they found the ratio was completely inverted.
The Pivotal Quote
"We found out actually model only maybe contributes 30 or 40% of the whole thing. And the framework, the whole system you build upon the model is much more important, you know, than the model itself." — Co-founder of the autonomous-agent startup
This statement landed differently than typical conference hot takes. It came with context: the startup began with the same assumption everyone has about model primacy. After 10 months of production deployment, real user feedback, and continuous iteration, they discovered the framework, the orchestration logic, and the system architecture built on top of the model contribute 60-70% to whether an agent actually works. The model itself, despite being the component that gets all the attention and investment, contributes a minority share.
This reframing has profound implications. If you are spending 90% of your time optimizing model selection and prompts while treating your surrounding infrastructure as an afterthought, you are optimizing the wrong thing. The competitive moat is not access to frontier models - it is the engineering that wraps around them.
Multi-Model Orchestration: Using Gemini for Search, GPT-4 for Code
One of the most practical insights from the fireside was how the startup actually structures their model usage. Rather than relying on a single "best" model for all tasks, they orchestrate different models for different phases of complex workflows:
"For some task the first phase will be gather information on Internet. And for that phase we will use Gemini Pro because similar model can access the Google index... And the second phase may be like writing some Python scripts to analyze the data... we will use like GPT-4 because GPT-4 kind of like the best Model for writing Python scripts. And in the third phase... we will use Claude because Claude is the best model to generate a very structured and a very beautiful layout HTML." — Co-founder of the autonomous-agent startup
This task-specific model selection represents a sophisticated understanding of model capabilities that most teams miss:
Phase 1 - Information Gathering: Gemini Pro handles web search and information retrieval, leveraging its integration with Google's search index. When your task requires pulling data from the internet, you want the model with the best search capabilities, not necessarily the highest benchmark scores.
Phase 2 - Data Analysis: GPT-4 takes over for Python script generation and data analysis. At the time, it was recognized as the strongest model for code generation, and the startup exploited this strength specifically.
Phase 3 - Output Generation: Claude handles the final output generation, particularly for structured HTML and slide layouts. Its strength in maintaining formatting consistency and producing visually coherent documents made it the right choice for this phase.
The insight here is not just "use multiple models" - it is that each model has genuine strengths that can be exploited through thoughtful orchestration. This requires more engineering complexity than single-model approaches, but delivers better results because you are matching model capabilities to task requirements.
The Orchestration Pattern
This multi-model orchestration pattern has emerged as a production best practice across our research. Teams that treat model selection as a one-time decision miss the opportunity to optimize at the task level. The framework that enables this routing - handling context passing between phases, managing state across model boundaries, implementing error recovery - IS the 60-70% that the startup identified.
8x Cost Reduction Through Architecture
Perhaps the most compelling evidence that framework architecture matters more than model capability came from the startup's cost optimization journey:
"I think it's really like 8 times cheaper compared to our first review. Our original expectation is that at the end of this year we're going to bring the whole system in price cheaper compared to March. But you know, we still have three months to go, but we've already achieved eight times cheaper." — Co-founder of the autonomous-agent startup
Eight times cheaper. Not from switching to a cheaper model - from architectural optimization. In just six months, they also achieved 4x speed improvements through system-level engineering.
This matters enormously for the economics of AI agents. Our research revealed that many agent startups charge $20-30 per month while consuming $5 or more in token costs per complex task - economically unsustainable at scale. The startup's 8x cost reduction demonstrates that the path to economic viability runs through framework engineering, not waiting for cheaper API pricing.
What "System Optimization" Actually Means
The architectural decisions driving these cost reductions include: intelligent caching to eliminate redundant API calls, strategic context compression (our research found agents using more than 40% of context windows start making mistakes), task-specific model routing to use cheaper models where appropriate, and parallelization of independent subtasks. The startup's "wide research" capability can execute 500+ concurrent simple tasks in 10-15 minutes - something only possible with sophisticated parallelization.
Each of these is a framework decision, not a model capability. The startup's trajectory - from $20/month prosumer pricing toward $2,000/month power user tiers - only becomes feasible when unit economics work. That requires architecture, not just better models.
Why This Matters
The Model Commodity Thesis
If models contribute only 30-40% to agent success, the corollary is that models are commoditizing faster than the industry realizes. GPT-4, Claude, and Gemini are increasingly interchangeable for many agent tasks. The differentiation has shifted to the surrounding infrastructure.
This explains a pattern we observed repeatedly in our research: teams that obsess over model selection often underperform teams that invest in framework engineering. The latter group treats the model as one component among many - important, but not sufficient.
Why OpenAI Agents Did Not Kill the Startup
When OpenAI launched ChatGPT agents, the startup's team was "the happiest person on earth" - it validated the market and allowed side-by-side comparisons showing their product outperforming. The model labs do not have an inherent advantage in the agent layer because the model is the minority contributor to agent success.
Building agents is "not just about the model" - OpenAI is "at the same starting line" as agent startups when it comes to the 60-70% that is framework architecture. Agent startups compete on framework engineering, user experience, and domain specialization where model providers have no structural edge.
The AI-Native Development Workflow Inversion
Beyond the core insight about model contribution, the fireside revealed how framework-first thinking changes how teams build products:
"Right now we are working like this. It's like when we have some ideas, we will just throw all our ideas to our agent. And then the agent will build the first prototype for us and we try to play with it... And then we will go to engineers, not designers." — Co-founder of the autonomous-agent startup
The traditional flow (PM creates requirements, designer creates mockups, engineer builds) has inverted. Their codebase shifted from 20% AI-generated code one year ago to 80% today. Product managers now push production PRs using Claude Code without engineer involvement for simple changes. This transformation is only possible when framework architecture - not model capability - determines velocity.
What You Can Do
Based on the fireside insights and corroborating evidence from our broader research, here are actionable recommendations for teams building AI agents:
Rebalance your investment ratio: If you are spending 80% of engineering time on prompt optimization and model selection, shift to 40% model work and 60% framework architecture.
Implement multi-model orchestration: Map your agent's tasks to phases and identify the best model for each phase. Start with three models: one for information retrieval, one for code/analysis, one for generation/formatting.
Treat cost optimization as architectural work: Build comprehensive logging to identify cost hotspots. Target architectural changes (caching, compression, model routing) rather than waiting for API price drops.
Build custom evaluation on real tasks: Industry benchmarks are "not even close to real user's requirements" per the startup. Create evaluation datasets from actual user tasks, segmented by vertical.
Design for model-agnostic flexibility: Build your framework to swap models per task phase without major rewrites. The framework that enables rapid model swapping is competitive advantage.
The Bottom Line
The fireside delivered a simple but profound insight: models contribute 30-40% to agent success, while framework architecture contributes 60-70%. This inversion of the industry's conventional wisdom has massive implications for how we build, invest in, and compete in AI agents. The competitive moat is not model access - it is the engineering that wraps around the models.
Stop waiting for better models to solve your agent problems. Start investing in the framework, orchestration, and system architecture that contributes the majority of agent success. The teams that internalize this insight - investing in multi-model routing, system-level cost optimization, and architectural patterns - will outperform those chasing the next frontier model release. The starting line is the same for everyone. The race is won in the 60-70%.
This post is part of my research series on AI Agent deployment, based on 36 expert interviews, 5 industry conferences, and 3 functional prototypes. Read the full research overview.
Have thoughts on AI agent deployment? Connect with me on LinkedIn or email me.