Can GPT improve digital transformations’ appalling failure rate?

70% failure rate

The research across business transformation and IT projects shows an appalling success rate. This is based on research covering multiple years, by different groups, looking at at 1,000s of projects. The data shows a very worrying trend. Now every organization relies on applications to drive its business, with less human oversight. GPT and AI will increase this emphasis. The chart below shows 26% complete failure, 44% sub-optimal results, and only 30% deemed a success.

This is an appalling statistic.  If this were the construction industry we would never cross a bridge or take the elevator up a skyscraper. If it were the airline industry, I wouldn’t have got into the plane – where I am writing this article – to fly to the UK. The pharma industry? We’d never take our medicine.

Here are some of the headline figures from the research. They all tell a familiar story.

• 87.5% fail to meet objectives: 3 Stages of a Successful Digital Transformation. Harvard Business Review,  20 September, 2022. Didier Bonnet
• 26% deemed ‘total’ failures: Flipping the Odds of Digital Transformation Success. Boston Consulting Group, October 29, 2020 Patrick Forth, Tom Reichert, Romain de Laubier, and Saibal Chakraborty
• 69% wasted spend: Digital Transformation Is Not About Technology. Harvard Business Review, March 19, 2019 Behnam Tabrizi, Ed Lam, Kirk Girard, and Vernon Irvin

This has not gotten better over time. It has gotten worse. We’ve improved the way that we build systems by more productive development languages. Assembler was difficult to wrtite. Procedural languages like COBOL, Fortran, and Basic made it easier (BTW Basic is 60 years old).  Object-oriented languages like C++, Java, and JavaScript gave another improvement. LowCode enabled non-technical people to develop applications.

Changes in methodology have not helped. The move from waterfall to agile has arguably made it far worse. That is not to say that the principles of agile are not well-founded, but agile is rarely implemented correctly. Sadly, Agile has come to mean “get it wrong but iterate to the right solution”. This was never the idea. Agile was intended to run shorter sprints to deliver the correct functionality. The shorter timeframes meant that the project could be more aligned with the changing business needs and priorities.

GPT offers further productivity step-changes. But will this make a difference to the overall objective: successful business transformation? When I ran major business transformation programs at Accenture, my definition of success was “On target, on budget, delighted client, and the team will work for me again.”

Why is it so bad (compared to other industries?)

Now, more than ever, the reason for project failure is that the underlying applications that support the business transformation do not deliver what the end users need. So let’s look at some of the issues that we are seeing and compare it to other industries:

Process is not regulated: There are no 3rd party, external checkpoints throughout the project, or at the end. Often the project process is poorly defined and not understood by the stakeholders. Even the smallest residential construction project requires an architect and local regulatory sign-off before construction starts. And then a final sign-off at the end of the project. The pharma drug testing is highly regulated.

The results are difficult to measure: The expected results were difficult to define at the outset, and there are many external factors that could impact the overall result. For example, the project may be an updated CRM system to improve sales closure. But external factors are the market, the competition, and changes in sales team or sales leadership. These all impact the results. There was a business justification and ROI case that was used to launch the project. Rarely do we see formal earned value or post-implementation reviews. With a construction project it is very clear if the project got delivered correctly.

Rush to deliver: The pressure to deliver quickly to enable the business to stay competitive means that shortcuts are taken, which ultimately delays getting results: insufficient analysis, limited architectural design, and inadequate documentation. These would all be eliminated if the project delivery process was rigorous and regulated. Pharma was considered a very slow moving industry with years to get a drug to market. COVID showed that the industry could change, without compromising patient safety. There were still the regulated checks.

Bad news is easy to bury: A project may not be a total failure resulting in a high-profile cancellation. The delivered application may take several iterations to get it right so that the users will adopt it. Let’s call that “noisy waste”.  It is clearly a wasted effort that is visible. This is a failure, but can be explained away (incorrectly) as “the nature of the agile approach”. There is also “quiet waste” that manifests itself as applications or functionality that is delivered and just never used. Again, a failed construction project if very difficult to hide (or bury). 

No project process improvement: The project is iterative, meaning that it is a series of small projects. So the project process is being repeated, and there is an opportunity to identify small improvements so that the delivery gets incrementally better. At Elements.cloud we review every software release to improve the process. We delivered 800 releases last year, each with major functionality changes, and we’ve optimized the process. This is why we were able to deliver so many releases with no rollbacks.

Root cause

Analysis of the root cause of project failures shows some clear trends. The top 3 are due to failings in the project delivery approach which lacks rigor driven by the desire to deliver quickly. Ironically the shortcuts taken to get results mean that the chance of success drops rapidly and overall it takes longer with huge levels of rework.

Business analysis: Insufficient time is spent to do enough rigorous analysis to validate the true business requirements. Too often the development team rush to deliver a solution based on the scant knowledge of the needs. It is not all their fault. Often the business does not really know what they need. They often describe it as what they want. This is not what they need. Good business analysis will tease out the need from the want. And only when the true requirements are understood and the implications of delivering it are understood (time to deliver, technical risk, regulatory impact, operation changes) can the delivery phase be started.

Architecture: The technical architecture is very difficult and extremely costly to change once a solution has been delivered. It has a lasting impact on the performance and usability of the application, the ongoing security, and the total cost of ownership based on the levels of technical debt. Architecture must be considered at the design phase. We know this from construction and manufacturing projects. But we don’t see enough projects where they have an architect.

Documentation: Time taken to document at different phases in the project is seen as wasted effort. Or if the value is understood, the pressure to deliver quickly means that it is never done. Documentation is a “gift to your future self”. It is delayed gratification. Great documentation helps you accelerate the next project because impact assessment is faster and more accurate. You are not starting from ground zero every time.s

How GPT can help

Whilst GPT can accelerate the development process, that is not the issue here. It is very clear from all the research, that the problems are before the development starts. The business analysis or planning phases are what is being missed. The rush into development is the problem to be fixed. Development cannot make up for failed analysis.

The fundamental issue is that teams jump from poorly understood requirements straight into development, following the red line in the image below. The challenge is to persuade the business that more time taken in the purple boxes will deliver the results that they need – but faster with less cost.

GPT can give 100x productivity improvements during the business analysis phases. It can suggest the questions to ask the business users. It can help analyze long interview transcripts and draw process diagrams. But these process diagrams are a first draft. They need to be reviewed and refined by the business analyst. GPT is not yet a replacement for a business analyst, but it is a valuable and massively productive assistant.

GPT can take the process diagrams and write the user stories with acceptance criteria. This is not only a huge time saver, but it also increases the accuracy of the user stories. As the user story is the final deliverable that is passed to the development team, it is the last chance to ensure that the project delivers the right solution. When GPT writes user stories and they are reviewed by the business analyst, and they are not accurate, then this points to a problem with the process diagram that was drawn. Because GPT writes the user stories so quickly, you can delete them update the process diagram so it better reflects the business, and get GPT to recreate the user stories. Before GPT you would write user stories and never go back and improve the process diagrams. And they are used for UAT and training so they need to be correct.

GPT can even recommend solutions based on the configuration of existing systems, taking into account architect best practices. This helps reduce technical debt because it suggests reusing existing metadata. The better your metadata description, the better the recommendations.

Sadly GPT cannot assist much in the creation of metadata description. This may feel surprising. GPT is great at creating text. It can document what something is, or what it is doing.  For example, a Flow or Apex. But the description needs to be “Why” something was done: Why you made that compromise decision, Why you decided on a particular way to solve the business problem. This has to be created by the person or team who made the decisions. GPT is very poor at trying to second-guess this. We just need to make it easier to document “in the moment” and keep documentation up to date. Here is a standard for metadata descriptions – MDD

GPT buys time for business analysts

Every business analyst (BA) knows that they are cutting corners. If they have more time they would do a better job. Often they lack the evidence and seniority to push back to get the time to do the analysis properly. We are already seeing Change Intelligence Platforms using GPT to take some of these time-consuming tasks and deliver them faster with a level of accuracy that makes the BA more productive. This will give teams the time to run a more rigorous process. That gives them the time to do better business analysis. They can then demonstrate the value of the approach to the business user, so they are allocated more time in future projects.

It is a virtuous circle. 

Can Salesforce buck the trend?

Salesforce is a platform that makes it very easy to configure. A key selling point is the ability to change it quickly – agility. Business transformation is more than just an application. It is getting users to change their working patterns, supported by changes to the underlying applications.

It is innovative, considered the top low-code platform, and is now leading the AI revolution. But as we’ve seen over the last 20 years, each improvement in delivery capability has not changed the success metrics markedly. 

The root causes of poor business transformation exist in the Salesforce ecosystem. Arguably they are even more prevalent as the declarative approach makes it even easier to jump to development without sufficient analysis. However, these issues are recognized. Salesforce has committed to improving business analysis. It has launched a BA Certification and training. It has developed a Well-Architected set of guidelines and is investing to promote this. There are a number of practitioners who are working hard to promote good business analysis principles. 

There are also software platforms that integrate with Salesforce that enable business analysis documentation to be created, analyzed, and reused. Elements.cloud is the most complete “Change Intelligence Platform”. There is a clue in the name. It is a platform that provides the intelligence to drive change faster. That intelligence comes from the interconnection between the business analysis artifacts (requirements, process maps, architect diagrams, ERD, user stories) and the systems configuration documentation. Now Change Intelligence Platforms are infused with AI, making business analysis work, faster and more accurate.

The Elements.cloud Change Intelligence Platform, ensures your development teams are building the right stuff, the first time, faster. Connect with the Elements team today and discover why our platform is the ideal partner for business analysts.