Microservices and Unicycling
After the unspeakable NoSQL hype of about two years ago had reached its peak “Why are you still working with relational databases?”, the topic of microservices was brought to the forefront when discussing back-end technologies. In addition, with React, Node & Co., the front-enders have developed quite a unique little game that, it seems, nobody else can see through.
After about two years of Spryker, I have had the pleasure of being able to follow these technical discussions first-hand. During my time with the mail order giant Otto Group, there was another quite clearly defined technical boogeyman — the so-called Host System, or the AS400 machines, which were in use by all the main retailers. Not maintainable, ancient, full of spaghetti code, everything depended on it, everything would be better if we could be rid of it and so on and so forth — so I was told. On the other side were the business clowns — I’m one, too — for whom technology was just a means to an end. Back then, I thought those who worked in IT were the real hard workers, pragmatic thinkers, who only answered to the system and whose main goal was to achieve a high level of maintainability. Among business people there were, and there still are, those I thought only busied themselves with absurd strategies and who banged on about omnichannel, multi-channel, target group shops and the like. Over the last eight years of Kassenzone.de, these strategies were always my self-declared final boss. It was my ultimate aim to disprove them and demonstrate new approaches.
To my great disappointment, I have to realize that people in IT — or ‘developers’ as they are called today — work with the same thought processes as the business clowns. There is an extremely high tendency to chase after trends and basic technical problems are not sufficiently analyzed, nor are they taken seriously enough. Microservices is a wonderful example of this. It is neither an IT strategy, nor is it an architecture pattern. At most it describes just a type of IT and system organization. Just like omnichannel. Omnichannel doesn’t mean anything. It’s a cliché that is pretty much just filled with “blub” and the same way of thinking is apparent on the topic of microservices. From the outside, omnichannel can be seen as the result of strong growth if a company’s range of services can, therefore, cause it to become a leader in many channels. This is exactly what happens with microservices, which may be the result of strong growth in IT, because you have to share large applications in services so that you don’t have too many developers working on them at the same time. But this is far cry from being an IT strategy. In many conversations at the code.talks conference, this impression was (unfortunately) confirmed. Yoav Kutner (founder of Magento1) cut his teeth at the roll out of the first of the big Magento1 projects and reports with a shake of the head that developers always follow the next hype without having considered where the real problem lies. Yes, I know that that sounds all very general, but let’s have a closer look at the topic of microservices.
Martin Fowler, IT guru and champion of microservices has written dozens of articles on the subject and describes microservices as follows:
"In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and are independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies."
This does sound quite promising and it can also help with the corresponding problems. Otto’s IT team has already reached the Champions League where this is concerned and produced the obligatory article, called “On Monoliths and Microservices” on the subject. Guido from Otto also referred to this topic at the code.talks event.
"When we began the development of our new Online Shop otto.de, we chose a distributed, vertical-style architecture at an early stage of the process. Our experience with our previous system showed us that a monolithic architecture does not satisfy the constantly emerging requirements. Growing volumes of data, increasing loads and the need to scale the organization, all of these forced us to rethink our approach."
There are also other examples which benefit excellently from this approach. Zalando is an example of a company which is open about using it in “From Jimmy to Microservice”. The approach can also crop up for quickly growing tech teams, such as that of Siroop. What’s often forgotten when people sing its praises are the costs associated with such an approach. Martin Fowler calls these costs the Microservice Premium and clearly warns against proceeding in this direction without caution.
"The microservices approach is all about handling a complex system, but in order to do so the approach introduces its own set of complexities. When you use microservices you have to work on automated deployment, monitoring, dealing with failure, eventual consistency, and other factors that a distributed system introduces. There are well-known ways to cope with all this, but it’s extra effort, and nobody I know in software development seems to have acres of free time. So my primary guideline would be don’t even consider microservices unless you have a system that’s too complex to manage as a monolith."
(Source: Martin Fowler)
Technically speaking, this restriction has many different origins. Whether it’s the latencies which must be called up for one procedure due to dozens of active services, clearly difficult debugging, the demanding hardware setup or the complex data retention (each service has its own database). The fundamentally hard to manage technology zoo notwithstanding. Here, excellent parallels to e-commerce organizations can be drawn. Who is quicker and more effective in the development and scaling of new models:
1. a company with dozens of departments and directorates, which must be in a permanent state of agreement, but which are extremely good in each of their individual disciplines;
2. or a company at which up to 100 employees sit in one room, all knowing what’s going on and all talking to each other.
The second example is quicker, that goes without saying. With larger models, at which scaling is a rather uniform approach, the first example is better. Not much is different in the case of microservices. To understand this context better, Werner Vogels’ (Amazon CTO) test on his Learnings with AWS is highly recommended.
"We needed to build systems that embrace failure as a natural occurrence even if we did not know what the failure might be. Systems need to keep running even if the “house is on fire.” It is important to be able to manage pieces that are impacted without the need to take the overall system down. We’ve developed the fundamental skill of managing the “blast radius” of a failure occurrence such that the overall health of the system can be maintained."
Although there are, therefore, really good guidelines for sufficient handling of the topic, so as to find out whether microservices make sense for an IT organization (not for most!), you can regularly find contributions such as that by Sam Gibson online or on conference panels:
"In principle, it is possible to create independent modules within a single monolithic application. In practice, this is seldom implemented. Code within the monolith most often, and quickly, becomes tightly coupled. Microservices, in contrast, encourage architects and developers the opportunity to develop less coupled systems that can be changed faster and scaled more effectively."
In Kassenzone reader’s language, this pretty much means: Pure Play business models are good if they are implemented in an orderly fashion but it will only really come good if you run many channels well. The winning strategy is omnichannel. Now, you could simply brush such statements off, but it is astounding just how quickly and strongly such simple thought processes spread and become the truth all by themselves. The voices which oppose them are quiet in comparison, but the arguments are quite conclusive.
"Finally, a word about nomenclature. Some advocates of micro-services like to classify the alternative as monolithic. This is a pejorative term chosen to imply: “Bad”. The word monolith means “one rock”. The implication is that if you aren’t using micro-services, then you must have a big coupled monstrosity. That’s Marketing Baloney. A well designed system following the Clean Architecture is about as far from a monolith as you can get."
Technically and methodically, a lot speaks for the use of “good” monolithic structures for a great deal of e-commerce companies, but doing so requires a lot of effort producing good code, something which, in the short term, you don’t have to do in the microservices world. If, then, a mistake in the scaling arises, the affected CTOs would probably wish they had the AS400 system back.
The founder of Basecamp has hit the nail on the head with his own system, which he describes as “The Majestic Monolith”. And, where content is concerned, I’m with him:
"Where things go astray is when people look at, say, Amazon or Google or whoever else might be commanding a fleet of services, and think, hey it works for The Most Successful, I’m sure it’ll work for me too. Bzzzzzzzz!! Wrong! The patterns that make sense for organizations’ orders of magnitude larger than yours, are often the exact opposite ones that’ll make sense for you. It’s the essence of cargo culting. If I dance like these behemoths, surely I too will grow into one. I’m sorry, but that’s just not how the tango goes."
It’s bit like if companies who own an old bicycle, which they don’t know how to ride properly, want a little too much. They see the unicyclist at the circus performing dazzling tricks on his unicycle and say to themselves: My bike is too old, that’s why I can’t ride it. I’ll just start with a unicycle right away, at least that’s forward-thinking.
So much for the observations of a non-IT bod into the world of the developer. At least the multichannel lie has already been disproven. Who’d have thought that Otto would be dominating the world of the unicyclist right now?
Keen to read more about the topic? How about some hot tips from Spryker CEO Fabian Wesner about "How NOT to get rid of a Monolithic Legacy E-Commerce System (with Microservices)"?