Agile Development and Core Systems White Paper


According to “the average median reaction time is 215 milliseconds” so shouldn’t your business react as fast as you do? We believe it should but the seismic shift in the availability of mobile, cloud and social channels has caused many businesses to fall way behind in the delivery of what the market and their target consumers are demanding. This paper discusses why this is the case and what the solution will be to ensure your business can react as fast as you do!

Pre Agile / Traditional development

Up until the agile revolution, IT projects were traditionally large, expensive and normally involved many technical people and resources to deliver them. A project would start with some initial discussions between systems analysts (or similarly titled people) who would discuss the requirements with the business to understand what must be delivered. Once all requirements were understood, discussions could start with developers and the eventual result would be a project plan detailing what would be delivered by whom and when.

Generally speaking such projects took months or years to deliver which was considered reasonable at the time and could take into account integration requirements with existing systems. There were many issues with this approach leading to many failed projects (that no one wants to talk about) such as:

  • Scope creep: As the project went forward, further requirements were added raising complexity to what were already large and complex projects.
  • Technology challenges: The lack of integration standards in the 20th century made integration between different systems and technologies very difficult leading to further potential for project delays.
  • People challenges: over the period of the project, people could leave and be replaced leading to loss of project knowledge and a need to skill up new people for the project.
  • Business challenges: even in the past, business requirements changed constantly so requirements identified at the start of a project may no longer be relevant and were replaced by other requirements which had to be dealt with.
  • Project challenges: having such large and complex projects with long elapse times between deliverables lead to project management and governance issues.

Obviously projects were successful over time but in many cases they over promised and under delivered or were simply scraped as it was felt there was little chance of success or time just caught up with them

What has Changed?

The entire landscape has effectively been turned on its head and changed utterly. Up until the early 2000s, companies spent large amounts of money building walls around their core systems to ensure that the data was only available internally within the organization and only to specific employees. Since that time, companies have been struggling to get this same data securely out to their customers (or B2B partners) on the web and now on mobile devices and even to the social networks.

This change has also heralded an empowerment of a new set of users: the customer. They are now far more demanding than any internal IT system user ever was and organizations are listening to them in order to retain or increase market share. The current generation of customers are not willing to wait months or even years for improved applications, they want to see constant improvements to the user experience and thus the applications they are using on their mobile devices, in the Cloud or in the social media.

This has led to a requirement to continuously and almost instantly react to new customer requirements and deliver new versions of applications to satisfy those requirements in days or weeks. This is almost the exact opposite to the traditional model which involved collecting all requirements in advance and then building to satisfy those requirements. An initial set of requirements is delivered with a new app in parallel with the collection of the next set of requirements while the priority of the delivery is determined in real time after the app has been released. This has led to the need for a new development paradigm which we now know as ‘Agile Development'.

Agile Development

‘Agile Development’ has at its core an ability to collect new requirements ‘on the fly’ and deliver on those requirements effectively in real time. This creates fundamental problems with the process of delivery as one must ask: “What requirements are valid requirements, what ones are ‘nice to have’ and what ones make no sense whatsoever”. Who can decide on this?

In many cases, the customer raising the requirement is the one who chooses. Perhaps a minimal implementation to address the requirement is delivered quickly and the usage of the new feature (or not as the case may be) will determine if it is retained, dropped or developed further. In other cases, multiple different ways of addressing the requirement are made available and the one that gets the best usage is retained with the others being dropped. In some cases, customers are asked to vote and the requirements with the most votes get addressed first. In a way, Agile Development has democratised the development process with the end users, and many more of them, getting a bigger say than they have ever had before.

How does this impact on core IT?

The processes that govern usage of core IT systems has been developed over the last 30 to 40 years and is very much based on doing things slowly and carefully to avoid expensive down time. There is normally a firm change procedure for delivering integration requests to provide data or application resources to new projects. In general, there is normally a long queue of requests that need to be dealt with which can result in new requests taking many months or even years to satisfy. This causes massive issues with the agile approach which needs rapid, cost effective access to the data or applications along with an ability to change them again if necessary almost weekly.

The integration models and related processes are normally based around 20thcentury integration frameworks from some of the large IT vendors which are very stable and very expensive; however, they have, over time, proven their worth by successfully supporting the running the business. The length of time these frameworks take to deliver means that by the time the requirements can be satisfied for a new agile project, the requirements may have changed or even disappeared in the intervening time. And even if they can be delivered in a reasonable length of time, any subsequent changes to deal with rapidly emerging requirements simply cannot be delivered in the time frames required.


How can Agile and Core IT systems get along?

Gartner have coined the term ‘Adaptive Integration’ which talks about adaptive frameworks which can complement existing integration frameworks. It is clear that the demand for ongoing integration work will not go away; however, using complementary technologies for agile projects will ensure that the IT people can deliver on requirements for agile projects alongside the more traditional integration requirements.

Adaptive integration with core IT systems offers the ability to deliver quick, cost effective integration for agile projects. This enables such projects to release new apps in a speedy way with costs that fit with the overall budget which would be impossible if the standard integration frameworks become part of that cost. They willalso ensure that the integration requirements can keep pace with the requirements being raised as new versions of the app are released daily if not weekly.

Ostia have many years of experience with core IT systems and understand their strengths and weaknesses. We have also had much experience with traditional integration stacks and the models that have built up around them but now offer an ability to provide adaptive integration solutions in the timeframe required for agile projects. Ostia are able to do this because of their Portus data integration technology they have developed over the last 10 years. It has taken 10 years for Ostia to get to the front of today’s technology. We can help you stay there.