According to Red Hat's Brian Gracely, success depends on "right-sizing" models via semantic routing and accelerating vulnerability response windows to under 14 days.
The honeymoon phase of the "chatbot" is over. It consumes tokens at an order of magnitude higher rate than static chat interfaces.
The Token Trap and the Rise of Semantic Routing
When an agent is tasked with resolving a simple insurance claim, it doesn't need the vast world knowledge of a top-tier LLM.
To combat this, sophisticated shops are implementing semantic routing. This is a layer of intelligence that sits in front of the LLM, classifying the intent of a user's request and routing it to the smallest, cheapest model capable of handling the job.
Now, the "FinOps" of AI requires teaching finance teams about token windows and context lengths.
Beyond routing, caching is the unsung hero of cost reduction. By storing the responses to repetitive queries, enterprises can bypass the GPU entirely for a significant percentage of their traffic, slashing latency and spend simultaneously.
The 14-Day Window: Why AI is Killing the Patch Cycle
Attackers are now using AI-powered discovery tools to find "chains" of minor vulnerabilities—flaws that seem harmless in isolation but become catastrophic when sequenced together.
According to Brian Gracely, the window to stay ahead of AI-driven exploits is shrinking to between seven and 14 days.
- AI-Driven Discovery: Automated tools are scanning for CVEs (Common Vulnerabilities and Exposures) faster than human teams can document them.
- The Embargo Crunch: The time between a vulnerability being discovered and its public disclosure is tightening, leaving defenders with almost no lead time.
- Strategic Patching: Rapid software management is no longer an “IT chore”; it is a strategic capability.
The Human Friction: Encoding Expertise Without Replacing Experts
For an agent to be useful, it must encode the nuanced, often undocumented knowledge of the people who actually run the business.
The goal is to automate the drudgery—the "caviar" tasks that don't require a human—while freeing the SME to handle high-value exceptions.
The Infrastructure Divergence
The dominant model providers are facing immense pressure to turn a profit after years of burning venture capital.
Stop worrying about how far behind you are.
