Over the past two years, organizations have been exploring how Generative AI can transform their business. The scale of change — whether evolutionary or revolutionary — varies from one enterprise to another, but one thing is clear: every organization is enhancing their tech stack to include GenAI.
At a fundamental level, the architecture of the modern IT stack hasn’t changed: microservices, event-driven systems, distributed workflows, … all supported by multiple teams. However, with the emergence of GenAI two significant shifts have occurred:
- Expanded capabilities: Components now handle vast amounts of unstructured, multimodal data, dramatically increasing the range of supported use cases.
- Reasoning and autonomy: Processes and workflows are increasingly shaped by language models interpreting context and determining the best sequence of actions, resulting in less deterministic and more adaptive behaviour.
In essence, while the longstanding challenges of developing and maintaining distributed systems persist, the advent of unstructured data and the emergence of autonomous capabilities add a new layer of complexity — and intrigue. A modern solution must efficiently handle diverse data sources and empower systems to reason independently, determining the best course of action to fulfil each request whilst assuring a safe and desired outcome.
If that wasn’t difficult enough, each business use case now also encounters a vast landscape of potential GenAI providers that are a ‘perfect’ fit. Some offer targeted solutions, like a copilot widget to organize your inbox, while others present comprehensive platforms for building and deploying GenAI agents, such as Agentforce.
Consequently, the next generation of IT platforms will have AI and GenAI at their core. They will be built from a diverse set of autonomous agents, seamlessly integrating across legacy and new systems, powered by multiple provider platforms delivered by multiple vendors.
Given this complexity, how should organizations approach the design of their IT infrastructure in the age of GenAI?
Based on hands on experience across industries, Accenture’s UK&I Centre for Advanced AI have authored this paper as part of a series exploring the challenges and potential solutions for delivering GenAI and agentic business solutions.
What are we aiming at? What is a future IT platform?
The future IT tech stack, driven by agentic AI, will be a multi-layered, interconnected ecosystem designed for autonomy, scalability, and continuous adaptation. It will have a composable architecture of specialized AI agents, with orchestration frameworks powered by Language Models (LLMs or SLMs). These agents will seamlessly integrate with existing enterprise systems via robust API and data mesh layers to solve challenging business problems.
AI agents’ capabilities, as illustrated in Figure 1, translate into significant business benefits including, increased efficiency, reduction in human error, improved customer experience, acceleration of innovation in product development, and enhanced scalability of operations.
What are the components we need to deliver these capabilities?
As touched upon previously, modern IT stacks in organizations are built on a diverse array of custom-built and commercial off-the-shelf (COTS) products, providing functionalities such as ERP, CRM, support and operations, HR, and more. Most of these tools offer some form of agentic capabilities, either through built-in agents or by providing a GenAI platform that enables the creation of new agents tailored to specific business processes. Therefore, in a modern IT stack (as illustrated in Figure3), agents will collaborate across each of these platforms and technologies, sharing data and requesting actions across all parts of the business to deliver comprehensive end-to-end use-case solutions.
Key Considerations for Successful Agentic AI Implementation
Agentic AI solutions share a set of challenges and considerations with more traditional AI solutions such as:
Data: High-quality, accessible, structured and unstructured data forms the backbone of any AI project. To achieve this requires good data architecture and engineering practices including standard data patterns and architecture, appropriate data governance, continuous data cleansing and integration efforts to drive out data silos and ensure data quality.
People and Change: Encompassing both skilled talent (data scientists, machine learning engineers, domain experts) and comprehensive AI literacy training for all employees. Addressing employee resistance and fears of job displacement through clear communication and upskilling programs is essential for successful adoption, in addition to an AI change management program.
Governance: Establishing clear AI governance policies covering model usage, data access, security, and ethical considerations is fundamental for responsible deployment. Organisations need to understand how, and when, these policies need to be applied throughout use-case inception and delivery processes to move at pace and ensure alignment throughout delivery (or stop delivery early).
The move towards agentic solutions extends and adds new areas of consideration:
- Strategy: What is the organizational strategy towards agentic AI? Within your business, where are the right places to be applying agentic solutions? Having a clear AI vision and roadmap, aligned with overall business goals is a must — understanding if you are evolving your existing business processes by replacing workflow steps with agentic solutions, or completely transforming your business with agentic technology at the heart? Creating a transformation / delivery approach that enables your organisation to realise value incrementally and quickly, instead of disappearing for 18mths to build a platform and then starting on use-cases, only to find that the core technology has changed underneath you.
- Tech Stack and Architecture: Agentic AI technology is moving at such a rapid pace that selecting a fixed set of technologies is going to be difficult, being careful not to get mired in Enterprise Architecture committees for weeks if not months of deliberating, given the breadth and frequency of change. Ensuring key architectural patterns are well defined and understood and then expecting that the underlying technologies will change is key.
- Assurance: Assuring AI solutions has always been difficult. Giving solutions agency makes this problem significantly more challenging — especially in a regulated environment, how do I ensure that my agent has followed the required process and utilized the source data appropriately to solve the request? Care must be taken through the solution design to truly understand where agency is appropriate and where it is not, and how that agency can be validated. What is my test strategy for agentic AI? What test data do I have to support it?
- Performance: Agentic workflows are inherently chatty to the LLM (and to external APIs but the LLM is the key component when it comes to performance). Because of this, latency and cost are key metrics to understand through design and into release. Optimising conversation flow with appropriate caching layers, tuned / compressed prompts, and appropriate model selection is required.
- Live Monitoring: Whilst our test strategy will attempt to assure the solution before release, it is only to be expected that the user (human or another AI agent) will throw in the odd curve ball resulting in some unexpected behaviour! Building solutions to support active monitoring will be key and providing the right observability and intervention capabilities when something unexpected happens is a vital design pattern.
Recommendations and Path Forward
Successfully adopting agentic AI at scale requires a deliberate and strategic approach. Based on our experience of delivering production workloads we propose the following recommendations to guide building a future-ready IT tech stack:
Strategic Approach: A Phased, Goal-Oriented Adoption
For most enterprises, a “big bang” IT transformation for agentic AI is risky. Instead, a phased, goal-oriented adoption strategy is typically a better approach:
- Start Small, Scale Smart: Initiate the journey with well-defined pilot projects that target specific transformational pain points and have clear, measurable objectives. This iterative approach allows for continuous learning and adaptation.
- Demonstrate Tangible Value: Focus on solutions that deliver value from a business perspective but also incrementally build out platform capabilities and organisational experience and skills. This builds momentum and secures broader organizational experience, understanding, and buy-in, across the business and technology domains which is critical for scaling.
- Embrace Agility: Take an agile approach to delivery. Be prepared to pivot as additional data, knowledge and experience are acquired through the delivery, agentic solutions are rapidly evolving it is likely that during delivery (or soon after) a new framework / model / pattern is released making part of the solution being delivered irrelevant.
Distribute from the centre but enable innovation across the organisation
Drive the development and adoption of agentic solutions through an AI CoE. Create a framework for innovation across the entire organization, enabling all parts of the workforce to contribute through code, PoV, hackathons, business insights, etc. Allow the CoE to curate innovation and align to organizational best practices to ensure a coherent agentic solution and then distribute across organization.
Engage with Vendors, Contribute to Interoperability Standards
The future IT stack will be multi-vendor and multi-platform, making strategic engagements and a focus on interoperability crucial.
- Strategic Vendor Collaboration: Partner with AI technology providers both commercial and open source aligned to your specific business goals and IT strategy. Use these relationships to guide your adoption of agentic AI solutions and to shape your partner’s roadmaps to meet your needs.
- Prioritize Interoperability: When selecting vendors for agentic AI solutions, evaluate their commitment to open standards and their support for emerging protocols like A2A (Agent-to-Agent) and MCP (Model Context Protocol). This will help to avoid vendor lock-in and will enable a composable, multi-vendor agentic stack.
- Active Ecosystem Participation: Consider actively participating in industry initiatives focused on AI interoperability. This shifts the enterprise role from adopter to ecosystem contributor, helping to shape the future of agentic AI standards and ensure they align with enterprise needs.
What’s next?
We hope this paper has provided a useful introduction to our thoughts on the future of IT systems and the initial steps towards adopting agentic AI. Lookout for our series of follow-on articles, where we will delve deeper into some of the concepts, components and capabilities touched on here, besides exploring in greater detail the key challenges associated with deploying agentic AI at scale and some proven strategies to mitigate them.