For rapid innovation, all types of modern businesses – enterprises to tech start-ups – all are increasingly adopting cloud platforms and microservice oriented architecture paradigms. This is making the overall coverage of a single application (or service in modern application language). This is leading to the natural need for integration of the micro applications within the enterprise.
Also, the boundary of the organizations is getting blurred. Increasingly, organizations are becoming a part of an extended ecosystem for business outcomes to end customers. This calls for integration with upstream and downstream business partners more effectively for the flow of information within the business ecosystem.
The Challenges
As internal and external drivers are making integration critical for modern businesses, leaders are looking more critically at the integration capabilities of the business. The questions they ask are:
“How integration can support a variety of integration scenarios with regards to application endpoints, Integration styles and can seamlessly connect modern as well as legacy applications?”
- How integration can support a variety of integration scenarios with regards to application endpoints, Integration styles and can seamlessly connect modern as well as legacy applications?
- How to support newer Integration requirements for microservices-based architecture, serverless and cloud-native solutions?
- How to keep pace with the agility of the engineering ecosystem where applications are now rolled out in hours and days instead of months as used to happen earlier?
- How integration mechanism and technology enable agility as well as the ability of customization as the primary goals as opposed to standardization, consistency, and reuse which ESB-based architectures typically used to focus on?
- Can integration support user personas and scenarios of integration in a user-friendly way, sometimes without much technical expertise?
- How to operate at Internet-scale with resilience and be responsive in real-time?
The Aspirations
When I interact with business leaders on their aspiration for the business enterprise, I come across a large number of aspirational elements:
- Our application ecosystem is being planned, built, managed to scale for the business need, integration should follow the application roadmap
- We need to own integration not only within our enterprise but look beyond the ecosystem of apps used by your customers and partners
- The enterprise front end must be able to connect back-office systems and make things happen
- The business processes span hundreds or even thousands of applications inside and outside the enterprise, all should be managed
- We need to know every single event happening in our application ecosystem so that we can analyze them and run our business more effectively
“We need to own integration not only within our enterprise but look beyond the ecosystem of apps used by your customers and partners”
Traditional ESB-style integration
There are few industries such as Telecom, Hospitality that still rely on a large number of COTS products and packaged applications. This trend has started changing in recent years as these organizations are trying to adopt modern architecture in building new capabilities or extending their existing capabilities. However, these industries will continue to have a large number of COTS products and packaged solutions on their portfolio. For integration, these organizations have invested in ESB-style platforms. The ESB-style platforms —either on-premises or in the cloud — fit their point to point, application to application integration requirements.
Modern message-oriented integration
Enterprises in the Retail and Financial Services industries have their IT landscape dominated by custom-built applications – in customer facing operations as well as in mid-office and back-office. Many of those applications are gradually modernizing to microservices-based architecture.
And in some new/redefined industries (smart health, smart home), applications are being built grounds up using modern architecture principles. These new applications implement some new functionalities but there is a heavy focus on integrating different applications into the ecosystem. For example, their medical device organizations are trying to bring out smart value-added services for their customers by integrating modern application components in the front with products in their traditional operations (say, medical diagnostics). The New Fintech industry is also creating a similar scenario for the Financial Service industry.
For these industry players, new application components are typically build using microservices architecture principles. And all those microservices are connected with internal/external application ecosystems through messaging middleware and API gateways.
For these industry players, as the architecture rolls out across their enterprise to retire the legacy applications or integrate with the legacy applications, these organizations increasingly are looking forward to ESB-style platforms giving way to Messaging middleware and API-oriented integrations.
Future of ESB
As the applications in enterprise ecosystem modernize, enterprises looks modern Integration platforms and architectures, ESB’s will face their own fate-
- ESB’s might away as enterprise applications are re-architected and modernized into a microservices-based architecture
- ESB’s might co-exist with new cloud-based Integration platform, as part of a Hybrid Integration architecture
- ESB’s will be modernized (re-factored) and deployed on a cloud platform
Future of Application Integration
As enterprises create and evolve their modern Application Integration strategy, the following should be key considerations:
- Have a modern event-oriented messaging middleware – built to communicate with today’s cloud-native, IoT and mobile technologies for modern microservices, event-driven architectures, or for the creation of an event mesh.
- Plug into your existing apps and infrastructure on-premises
- Accept events and messages from high velocity (thousands, millions of events every minute) event sources, e.g. – IoT devices, gateways, cloud apps and back-end systems
- Integrate your cloud-native apps and cloud services
- Connect with the containerized apps and your iPaaS
- It should enable robust and event-driven data transport
- Create hybrid integration strategies that evolve and add more integration capabilities over time. Ensure that different integration needs (event-based integration, ability to integrate with traditional applications, integrate with cloud native applications) are sufficiently addressed.
- Loose coupling of upstream and downstream systems and real-time Integration are critical capabilities in an agile organization’s vision for Integration. Make event-oriented architecture components e.g. – event brokers, event sourcing and serverless components – a key part of your integration
- Address the increasing adoption of microservices and related integration needs in the context of modernization. Explore cloud-native integration capabilities using service mesh and other microservices-based integration capabilities.
- Even though integration platforms play an important part of integration strategy of an enterprise, adoption of right integration patterns, practices in addition to vendor products make the integration design more robust
“Create hybrid integration strategies that evolve and add more integration capabilities over time. Ensure that different integration needs (event-based integration, ability to integrate with traditional applications, integrate with cloud-native applications) are sufficiently addressed.”