How NOT to get rid of a Monolithic Legacy E-Commerce System (with Microservices)

Fabian Wesner
Fabian Wesner CTO Spryker
21. September 2017 in

Technology English

When we talk to potential Spryker customers, we often find them struggling to run an e-business worth hundreds of millions of euros using a 5–15-year-old monolithic legacy system. Further development is often restricted due to the risk of unpredictable side effects. What’s more, the people who set up the legacy system have usually left the company, and the current team can only guess at how things work. The legacy system then quickly becomes a bottleneck for the company’s growth.

The “cut-into-services” approach

We often see our customers taking one of a number of similar step-by-step “cut-into-services” approaches. The initial idea is to divide the monolith into services over time. This way there is no “big bang”, but a smooth series of incremental shifts to a better system. This approach appeals to developers because it lets them to get hands-on experience with much-vaunted microservices. This often makes it a no-brainer for the management to opt for this approach, as it involves less risk, no need for a feature freeze, and is welcomed by developers. 

Approached solution

 

So, let’s have a look forward how these projects usually go

Everything is straightforward at first. There is usually a dedicated team that starts extracting services from the monolithic application. They always start with the loose ends of the system, and they want to make their own decisions about the programming language and other technologies. Soon, the monolith is surrounded by satellite services, such as a mail service, a payment service and an invoice service. There is a UML-like target-architecture diagram and the use of buzzwords and jargon, such as “ESB”, “service discovery”, “database clusters” and “full-JavaScript front-end requiring server-side pre-rendering”. It’s a perfect playground for developers, with tons of interesting infrastructure challenges and new technologies. Sadly, these diagrams are never updated and become useless in next to no time.

Typically, the team has no experienced software architect to deploy a service-oriented architecture successfully, so the team is in “learning-by-doing” mode. Some developers are aware of the prerequisites for microservices, but it’s mentally so much easier just to start coding, and that is what usually happens. After a few months, it becomes obvious that the newly created services have differences. You’ll see several programming languages, or at least different versions of the underlying frameworks. There’ll be a lot of redundant development going on, and the team will be fighting with asynchronous communication including outages and latency. Simple object calls or joins in the database become requests through the network and require costly cross-team contract negotiations in advance.

After a year or two, some of these challenges will have been resolved but new ones will have arisen. The monolith will still be there, and will not even have shrunk as expected. This is because business continued at full throttle throughout the project, so there was continuous development going on in the legacy system. Actually, instead of “just a dirty monolith”, you will now have a monolith plus a bunch of service satellites that don’t follow any conventions. Instead of reducing complexity, you will have dramatically increased it.

The result after some time

What happens next?

The bad news is that, from this point on, it becomes harder and harder to extract services from the monolith. This is because all e-commerce systems are conceptually coherent. The domain objects have relations that cannot simply be removed. While it’s not that hard to remove coupling in code, it’s a very tough job to do the same with data relations. Actually, in almost every e-commerce system I have seen, 90% of all objects are coupled and cannot be separated in an iterative approach.

Take the example of a discount service. This is a typical example that was so easy to draw as a service on the architecture diagram. But when you look inside your database, you’ll see all sorts of cross-boundary relations. Single discounts are only valid for products in a specific category. Voucher codes may have a customer balance. They don’t apply for some products, such as gift cards or products that have already been reduced by a catalogue price rule. They may not be compatible with some payment methods. They are deeply integrated into your cart and checkout calculation and they need to be obeyed in the case of refunds. So just this one service comes with a lot of (conceptual!) dependencies and makes technical decoupling very hard. In any case, it’s almost impossible just to pull out discounts while keeping the rest of the system intact.

To make things worse, over time you’ll need to fight brain drain, because some of your developers will find new jobs and leave you alone with your brand-new and expensive distributed monolith. Good luck!

 

What's an alternative approach?

Proven, best-practice scenarios include a number of factors that can be tackled by a business of any size. Curious to find out how you can master replatforming and abolishing your monolithic legacy e-commerce system? You can find some tips here.

 

Still got questions?
Ask the author for further information.