04 Jun 16 FAMOUS ERP DISASTERS, DUSTUPS AND DISAPPOINTMENTS
It’s no wonder ERP has such a bad reputation: The history surrounding the complex and expensive enterprise software market is packed with tales of vendor mud-slinging, outrageous hype and epic failures.
With enterprise resource planning (ERP) and customer relationship management (CRM) applications at the heart of many a company’s operations, the consequences of a failed software rollout can be serious, including shareholder lawsuits and financial meltdown. But after a spate of high-profile failures, there are signs that vendors and customers alike are working hard to ensure the success of their ERP projects. Panorama Consulting Solutions, which regularly surveys businesses on the outcomes of their ERP projects, found in 2015 that just 58 percent of organizations rated their latest project a success, while by 2019 that figure had risen to 88 percent.
However, the success rate was somewhat at odds with the number of projects said to be overrunning or underperforming. As Panorama wryly noted, “It appears that organizations have lower standards of success than they probably should.” It may be, too, that companies wish to avoid the reputational damage that comes from failure, and instead prefer to redefine success as whatever they get. Sometimes the only sign something has gone wrong is when the parties head to court- and the full details of the dispute rarely come out.
Greg Crouse, managing director at Navigant Consulting, has learned all about this from inside the belly of the litigious beast, serving as an expert court witness or consultant after spending 25 years managing large-scale projects. “You’d have a hard time finding anyone who will talk about it- cases either litigate forever or get settled and sealed,” says Crouse. Nevertheless, we’ve assembled some dramatic ERP flops from over the years and tried to suss out some wisdom from the wreckage. (All comments from Crouse are about his general experience with these kinds of cases; he hasn’t actually worked on any of the specific disasters we’re discussing here.)
1. Leaseplan: A monolith unfit for the emerging digital world
After an initially successful SAP deployment at its Australian subsidiary, in 2016 vehicle management company Leaseplan commissioned HCL Technologies to develop a new SAP-based Core Leasing System (CLS) that was to be the heart of the group’s IT transformation across 32 countries.
In early 2018, auditors warned of exceptions with respect to user access and change management in CLS, and recommended improvements to IT controls and governance as more countries were expected to migrate to CLS that year. By March 2019, things were slipping. The auditors noted that rollout of “the first phases” of CLS was now expected that same year, and added recommendations on managing outsourcing risk to their earlier warnings.
Leaseplan abandoned CLS months later, writing off €92 million ($100 million) in project costs, and millions more in related restructuring and consultancy fees. It managed to salvage just €14 million it had spent on separately developed IT modules that it expected would generate economic benefits in the future.
The problem, Leaseplan said in notes to its second-quarter results, was that CLS would “not be fit for purpose in the emerging digital world in which [it] operated.” The monolithic nature of the SAP system “hindered its ability to make incremental product and service improvements at a time of accelerated technological change,” according to Leaseplan.
Instead, the company planned to build a modular system using best-of-breed third-party components alongside its existing predictive maintenance, insurance claim and contract management systems. It expected this to be more scalable and allow incremental product deployments and updates.
2. MillerCoors: Fighting in public, then making nice
In 2014, MillerCoors was running seven different instances of SAP’s ERP software, a legacy of the years of booze industry consolidation that had produced the alcohol behemoth. The merged company hired Indian IT services firm HCL Technologies to roll out a unified SAP implementation that would serve the entire company. Things did not go smoothly: The first rollout was marked by eight “critical” severity defects, 47 high-severity defects, and thousands of additional problems recorded during an extended period of “go-live hypercare.” By March 2017 the project had gone so far south that MillerCoors sued HCL for $100 million, claiming HCL had inadequately staffed the project and failed to live up to its promises.
But the IT services company didn’t take that lying down: In June of 2017 HCL countersued, claiming MillerCoors was in essence blaming HCL for its own management dysfunction, which HCL said was at the real cause of the failure. Outside observers noted that the wording of the contracts, as outlined in the lawsuits, seemed to be based on a pre-existing general services contract between the two companies, and left plenty of room for error. Then, in December 2018, the two companies resolved the dispute “amicably,” having apparently used the courts as a venue for a high-stakes, public negotiating session.
3. Revlon: Screwing up badly enough to enrage investors
Cosmetics giant Revlon was another company that found itself needing to integrate its processes across business units after a merger — in this case, it had acquired Elizabeth Arden, Inc., in 2016. Both companies had had positive experiences with ERP rollouts in the past — Elizabeth Arden with Oracle Fusion Applications, and Revlon with Microsoft Dynamics AX. But the merged company had made the fateful choice to go with a new provider, SAP HANA, by December 2016.
Was HANA an undercooked product doomed to fail? Maybe. What’s clear was that the rollout was disastrous enough to essentially sabotage Revlon’s own North Carolina manufacturing facility, resulting in millions of dollars in lost sales. The company blamed “lack of design and maintenance of effective controls in connection with the… implementation” for the fiasco in March 2019, and noted that “these ERP-related disruptions have caused the company to incur expedited shipping fees and other unanticipated expenses in connection with actions that the company has implemented to remediate the decline in customer service levels, which could continue until the ERP systems issues are resolved.” The crisis sent Revlon stock into a tailspin that in turn led to the company’s own stockholders to sue.
4. Lidl: Big problem for German supermarket giant
It was supposed to be the marriage of two great German companies: SAP, the ERP/CRM superstar, and Lidl, a nationwide grocery chain with €100 billion in annual revenue. The two companies began working together on a transition away from Lidl’s creaky in-house inventory system since 2011. But by 2018, after spending nearly €500 million, Lidl scrapped the project.
What happened? The scuttlebutt is that the problem centered on a quirk in Lidl’s record-keeping: They’ve always based their inventory systems on the price they pay for goods, whereas most companies base their systems on the retail price they sell the goods for. Lidl didn’t want to change its way of doing things, so the SAP implementation had to be customized, which set off a cascade of implementation problems. Combine this with too much turnover in the executive ranks of Lidl’s IT department, and finger-pointing at the consultancy charged with guiding the implementation, and you have a recipe for ERP disaster.
5. National Grid: A perfect storm
National Grid, a utility company serving gas and electric customers in New York, Rhode Island, and Massachusetts, was facing a difficult situation. Their rollout of a new SAP implementation was three years in the making and already overdue. If they missed their go-live date, there would be cost overruns to the tune of tens of millions of dollars, and they would have to get government approval to raise rates to pay for them. If they turned on their new SAP system prematurely, their own operations could be compromised. Oh, and their go-live date was November 5, 2012 — less than a week after Superstorm Sandy devastated National Grid’s service area and left millions without power.
In the midst of the chaos, National Grid made the fateful decision to throw the switch, and the results were even more disastrous than the pessimists feared: Some employees got paychecks that were too big, while others were underpaid; 15,000 vendor invoices couldn’t be processed; financial reporting collapsed to the extent that company could no longer get the sort of short-term loans it typically relied on for cashflow. National Grid’s lawsuit against Wipro, its system integrator, was eventually settled out of court for $75 million, but that didn’t come close to covering the losses.
6. Worth & Co.: Interminable rollout leads to a lawsuit at the source
Worth & Co. is a Pennsylvania-based manufacturing company that just wanted a new ERP system, and after hearing several pitches in 2014, decided to hire EDREi Solutions to implement Oracle’s E-Business Suite. The first go-live date was November 2015. But then things began to slip. The deadline was pushed back to February 2016; at that point Oracle demanded that Worth & Co. pony up $260,000 for training courses and support contracts. But 2016 came and went and still no rollout. In 2017 Worth & Co. jettisoned EDREi for another integrator, Monument Data Solutions. Another year was spent attempting, without success, to customize Oracle’s suite for Worth & Co.’s purposes.
Finally, after the project was abandoned, Worth & Co. did something novel in February 2019: They sued not their IT vendor, but Oracle, specifically citing the $4.5 million they paid the software giant for licenses, professional services, and training. The lawsuit is still ongoing.
7. Vodafone: The long arm of the law
When British telecom provider Vodafone consolidated its CRM systems onto a Siebel platform, they ran into problems: not all the customer accounts migrated properly. The company didn’t go out of its way to advertise this, of course, but people started to notice when their accounts weren’t properly credited for payments made.
The upshot: a £4.6 million fine from the British telecom regulator. And while this incident was concluded with just the fine paid, Crouse points out that regulatory oversight can, somewhat surprisingly, lead to private litigation down the road. “If there’s problems with large scale implementations, people are going to find out about it — because you have to report it to your regulator if things go bad.” Whereas a company might’ve been previously tempted to keep quiet about the whole affair, with regulators revealing screwups, that company might decide its best bet is to cast blame on someone else through litigation.
8. Washington community college system: When third parties flop
But that litigation can go both ways. For instance, students at Washington State’s community colleges have been paying a portion of their tuition every year to help the schools upgrade to a PeopleSoft ERP system that was supposed to go live in 2012. Instead, the project is still limping along. One cause of delay was internal: the 34 campuses in the system had widely varying business processes that needed to be standardized, which wasn’t clear until well into the rollout.
But now another crisis has emerged: Ciber, the third-party company hired to roll out the PeopleSoft system, went bankrupt in April of this year, only to have its assets scooped up by HTC, a Michigan company- and HTC then cancelled its contract with the school system and sued for $13 million, claiming the failed rollout was due to “internal dysfunction” on the colleges’ part.
Crouse says that this sort of mutual animosity is not uncommon. “You get into cases where the client is unhappy with the work the implementation firm has done and so they sue them. You also get into issues of the client’s not happy so they stop paying the bills. Then you have the third parties that sometimes get involved from a vendor reseller perspective. You can see either side being the plaintiff or the defendant, based on who got mad first.”
The rollout is meanwhile stuck in limbo.
9. Woolworth’s Australia: The death of institutional memory
The Australian outpost of the venerable department store chain, affectionately known as “Woolies,” also ran into data-related problems as it transitioned from a system built 30 years ago in-house to SAP. One of the biggest crises that arose was that profit-and-loss reports tailored for individual stores, which managers were accustomed to receiving every week, couldn’t be generated for nearly 18 months.
The problem lay in the change in data collection procedures, but the root cause was a failure of the business to fully understand its own processes. The day-to-day business procedures weren’t properly documented, and as senior staff left the company over the too-long six-year transition process, all that institutional knowledge was lost — and wasn’t able to be baked into the new rollout.
“I often see companies that don’t take the people who really know business processes and dedicate them to the ERP rollout,” says Crouse. “They make it a part-time job, or they hire new people to tell the system guys what to build. None of that works. You have to really dedicate the people who know the process that you’re trying to get right, full-time. And it’s a common theme that, when you don’t dedicate those people, you get into trouble.”
10. Target Canada: Garbage in, garbage out
Many companies rolling out ERP systems hit snags when it comes to importing data from legacy systems into their shiny new infrastructure. When Target was launching in Canada in 2013, though, they assumed they would avoid this problem: there would be no data to convert, just new information to input into their SAP system.
But upon launch, the company’s supply chain collapsed, and investigators quickly tracked the fault down to this supposedly fresh data, which was riddled with errors— items were tagged with incorrect dimensions, prices, manufacturers, you name it. Turns out thousands of entries were put into the system by hand by entry-level employees with no experience to help them recognize when they had been given incorrect information from manufacturers, working on crushingly tight deadlines. An investigation found that only about 30 percent of the data in the system was actually correct.
11. PG&E: When ‘sample’ data isn’t
Some rollouts aim to tackle this sort of problem by testing new systems with production data, generally imported from existing databases. This can ensure that data errors are corrected before rollout — but production data is valuable stuff containing a lot of confidential and proprietary information, and it needs to be guarded with the same care as it would in actual production.
In May of 2016, Chris Vickery, risk analyst at UpGuard, discovered a publicly exposed database that appeared to be Pacific Gas and Electric’s asset management system, containing details for over 47,000 PG&E computers, virtual machines, servers, and other devices — completely open to viewing, without username or password required. While PG&E initially denied that this was production data, Vickery says that it was, and was exposed as a result of an ERP rollout: a third-party vendor was given live PG&E data in order to fill a “demo” database and test how it would react in real production practice. They then failed to supply any of the protection a real production database would need.
12. Definitely not a sweet experience for Hershey’s
Could a failed technology implementation (in this case SAP’s R/3 ERP software) take down a Fortune 500 company (in this case Hershey Foods)? Well, it certainly didn’t help Hershey’s operations during the Halloween season in 1999 or make Wall Street investors thrilled.
In the end, Hershey’s ghastly problems with its SAP ERP, Siebel CRM and Manugistics supply chain applications prevented it from delivering $100 million worth of Kisses for Halloween that year and caused the stock to dip 8 percent.
So I guess a failed technology project can’t actually take down a Fortune 500 company for good, but it can certainly knock it around a bit.
13. Just do it: Fix our supply chain system!
What did a $400 million upgrade to Nike’s supply chain and ERP systems get the world-renowned shoe- and athletic gear-maker? Well, for starters, $100 million in lost sales, a 20 percent stock dip and a collection of class-action lawsuits. This was all back in 2000, and the horrendous results were due to a bold ERP, supply chain and CRM project that aimed to upgrade the systems into one superstar system. Nike’s tale is both of woe and warning.
14. HP’s ‘perfect storm’ of ERP problems
The epic tale of HP’s centralization of its disparate North American ERP systems onto one SAP system proves that one can never be too pessimistic when it comes to ERP project management. You see, in 2004, HP’s project managers knew all of the things that could go wrong with their ERP rollout. But they just didn’t plan for so many of them to happen at once.
The project eventually cost HP $160 million in order backlogs and lost revenue—more than five times the project’s estimated cost. Said Gilles Bouchard, then-CIO of HP’s global operations: “We had a series of small problems, none of which individually would have been too much to handle. But together they created the perfect storm.”
15. A new type of freshman hazing
Pity the college freshman at the University of Massachusetts in fall 2004: The last thing they needed was some computer program to haunt their lives and make their new collegiate experience even more uncertain.
But more than 27,000 students at the University of Massachusetts as well as Stanford and Indiana University were forced to do battle with buggy portals and ERP applications that left them at best unable to find their classes and at worst unable to collect their financial aid checks. Said one UMass senior at the time: “The freshmen were going crazy because they didn’t know where to go.” After a couple of tense days and weeks, however, everyone eventually got their checks and class schedules.
16. Waste Management trashes its ‘fake’ ERP software
Garbage-disposal giant Waste Management took SAP to court over an 18-month installation of its ERP software, seeking over $1 billion in damages. The initial deal began in 2005, but the legal saga commenced in March 2008, when Waste Management filed suit and claimed SAP executives participated in a fraudulent sales scheme that resulted in the massive failure.
Several months later, SAP fired back, claiming that Waste Management allegedly violated its contractual agreement with SAP in several ways, including by “failing to timely and accurately define its business requirements,” and not providing “sufficient, knowledgeable, decision-empowered users and managers” to work on the project.
The two went to arbitration in March 2010 and soon reached a deal, although the end result left neither happy: Waste Management later reported that it had received a cash benefit of just $77 million in settlement of the litigation, while SAP went on to sue one of its liability insurers, Swiss Re, for refusing to pay out on the settlement.
Surviving an ERP rollout
So what have we learned? Well, don’t fall afoul of regulators, make sure your data is secure and clean, and document your processes before you move to a new platform: all good advice for any rollout (or any other big IT project, really). If there’s one other key word Crouse has for CIOs, it’s this: continuity.
“I’m working a case today that involves an ERP implementation with a timeline of multiple years,” he says, “and there have been four CIOs during that timeframe. That causes a whole host of problems. You have to have an executive sponsor. You have to have someone who’s really championing the project. It’s difficult if the people at the top and the people who know the project from the client side continually change.”