When we began the development of our new Online Shop otto.de, we chose a distributed, vertical-style architecture at an early stage of the process. Our experience with our previous system showed us that a monolithic architecture does not satisfy the constantly emerging requirements. Growing volumes of data, increasing loads and the need to scale the organization, all of these forced us to rethink our approach.
Therefore, this article will describe the solution that we chose, and explain the reasons behind our choice.
In the beginning of any new software development project, lies the formation of the development team. The team clarifies such questions as the choice of the programming language and the suitable framework. When talking about Server Applications (the subject of this article), some combination of Java and Spring Frameworks, Ruby on Rails or similar frameworks are the common choices for the software teams.
The development starts, and, as a result, a single application is created. At the same time, a macro-architecture becomes implicitly chosen in the process without being challenged by anyone. Its disadvantages come slowly to the surface during further development:
- It gives rise to a heavyweight Micro Architecture
- The scalability is limited to Load Balancing
- The maintainability of the system often suffers in the case of particularly big applications
- Zero downtime deployments become rather complicated, especially for stateful applications
- Development with multiple teams is inefficient and requires a lot of coordination
…just to name a few.
Of course, no new development starts as a “Big Monolith”. In the beginning, the application is rather slim, easily extendable and understandable – the architecture is able to address the problems that the team has had up to this point. Over the course of the next months, more and more code is written. Layers are defined, abstractions are found, modules, services and frameworks are introduced in order to manage the ever-growing complexity.
Even in the case of a middle-sized application (for example, a Java Application with more that 50,000 LOC), the monolithic architecture slowly becomes rather unpleasant. That especially applies to applications that have high demands for scalability.
The slim fresh development project turns into the next legacy system that the next generation of developers will curse about.
Divide and Conquer
The question is, how can this kind of development be avoided, and how can we preserve the good parts of small applications inside the big systems. In short: how can we achieve a sustainable architecture that will after the years still allow for efficient development.
In software development, there are a lot of concepts around structuring the code: Functions, Methods, Classes, Components, Libraries, Frameworks and etc. All of these concepts share just one single purpose: help the developers to better understand their own applications. The computer has no need for classes – we do.
So, as all of us, software developers, have internalized these concepts, a question comes to mind: why should these concepts stay limited to just one application? What prevents us from dissecting an application into a system of several, loosely coupled processes?
Essentially, there are three important things to keep in mind:
- Conway’s Law: The development starts with a team, and in the beginning, the application is very understandable: at first, you have a “need” for just one application.
- Initial costs: Spinning up and operating a new application only appears to be an easy task. In reality, you need to set up and organize a VCS repository, build files, a build pipeline, a deployment process, hardware and VMs, logging, monitoring tools and etc. All of these concerns are often accompanied by certain organizational hurdles.
- Complexity in operations: Big distributed systems are clearly more complex to operate than a small load balancer cluster.
If we let things slide, we will generally not end up with a system consisting of small, slim applications, but with a big clunky monolith. And the biggest problems reveal themselves when it is already too late.
Normally, it is initially rather clear if a particular application has to be scaleable, and if a rather large code base is underway. It is then the case that when you come around the above mentioned obstacles, you either have to solve them, or live forever with the consequences.
At OTTO, for example, it did pay off to establish four cross-functional teams very early in the process, and therefore to prevent the development from belonging to just one team. As a consequence, four applications originated instead of just one.
As we already had to operate a big system in the past, the complexity of operations seemed a solvable topic for us: in the end, there is little difference in operating 200 instances of a monolith or a similar number of smaller systems.
Initial costs of spinning up the new servers can eventually be overcome through standardization and automation. Since we are not referring to cloud services, you indeed have to do some groundwork at least once in order to set up this automation. But then, you can benefit from all this work for a long time in the future – a cost that pays off.
So, how can we design a system made out of small, slim applications instead of a monolith? First, let’s remind ourselves in which dimensions applications can be scaled.
The vertical decomposition is such a natural approach that it can be easily overlooked. Instead of packaging all of the features into one single application, we disassemble the application into several applications that are as independent as possible of each other from the very start.
The system can be cut according to business sub-domains. For example, for otto.de we have divided the online shop into 11 different verticals: Backoffice, Product, Order and so on.
Each one of these “verticals” belongs to one single team, and it has its own Frontend as well as Backend and data storage. Shared code between the verticals is not allowed. In the rare occasions where we still want to share some code, we do so by creating an Open Source Project.
Verticals are therefore Self-Contained Systems (SCS), as they are referred to by Stefan Tilkov in his presentation about “Sustainable Architecture“.
A vertical can still be a relatively big application, so one might want to divide it even further: preferably by splitting it into more verticals, or by disassembling it into more components through distributed computing, where the components run in their own processes and communicate with each other via, for example, REST.
In doing so, the application is not only split vertically, but also in a horizontal way. In such an architecture, requests are accepted by services, and the processing is then distributed over multiple systems. The sub-results are then summarized into one response and sent back to the requestor.
The individual services do not share a common database schema because that would mean tight coupling between the applications: changing of the data schema would then lead to a situation where a service can no longer be deployed independently from the other services.
A third alternative for the scaling of a system can be appropriate when very big amounts of data are being processed, or when a decentralized application is being operated: for example, when a service needs to be offered worldwide.
However, as we are not utilizing the concept of sharding at this moment, I am not going to go into any further detail at this point.
As soon as the single server cannot process the load anymore, load balancing comes into play. An application is then cloned n-times, and the load is divided via a load balancer.
Different instances of load-balanced applications often times share the same database. This can therefore lead to a bottleneck, which can be avoided by having a good scalability strategy in place. This is one of the reasons why NoSQL DBs have established themselves in the software world in the recent years, since they can often deal with scalability in better ways than relational DBs.
All of the mentioned solutions can be combined with each other in order to reach almost any level of scalability within the system.
Of course, this can only be means to an end. When you do not have the corresponding requirements, the end result can become a little bit too complex. Thankfully, one can take small steps in order to bring themselves towards their target architecture, instead of starting with an enormous plan from the very start.
As an example, at otto.de, we started with four verticals plus load balancing. Over the last three years, more verticals came into being. Meanwhile, some grew to be too big and bulky. So currently, we are in the process of introducing microservices and extending the architecture of individual verticals in order to establish distributed computing.
The term microservice has grown very popular recently, especially when talking about the system that is being sliced into finer, more granular domains.
The term microservice has grown very popular recently. Microservices are an architectural style, where a system is being sliced along business domains into fine-grained, independent parts.
In this context, a microservice can be a small vertical, as well as a service in a distributed computing architecture. The difference to traditional approaches lies in the size of the application: a microservice is supposed to implement just a few features from a particular domain, and should be completely understandable by a single developer.
A micro service is typically small enough, such that multiple microservices could be run on a single server. We have had good experience with “Fat JARs”, which can be easily executed via java –jar <file> and, if needed, can start an embedded Jetty or a similar server.
In order to simplify the deployment and the operations of the different microservices on a single server, each server runs in its own Docker container.
REST and microservices are a good combination, and they are suitable for building larger systems. A microservice could be responsible for a REST resource. The problem of service-discovery can be solved (at least partially) via hypermedia. Media types are helpful in situations that involve versioning of the interfaces, and the independence of service deployments.
Altogether, microservices have a number of good properties, some of which are:
- The development in a microservice architecture is fun: every few weeks or months, you get to work on a new development project instead of working on an oversized old archaic system
- Due to their small sizes, microservices require less boiler plate code and no heavyweight frameworks
- They can be deployed completely independently from one another. The establishment of continuous delivery or continuous deployment becomes much easier
- The architecture is able to support the development process in several independent teams
- It is possible to choose the most appropriate software language for each service respectively. One can try out a new language or a new framework without introducing major risks into the project. However, it is equally important not to get carried away by this newly emerged freedom
- Since they are so small, they can be replaced with reasonable costs by a new development project
- The scalability of the system is considerably better than in a monolithic architecture, since every service can be scaled independently from the others
Microservices comply with the principles of agile development. A new feature that does not fully satisfy the customers cannot only be created in a fast way, but it can be also destroyed as quickly.
Macro- und Micro-Architecture
The term software architecture traditionally implies the architecture of a single program. In vertical or microservice style architecture, the definitions like “Architecture is the decisions that you wish you could get right early in a project” is hardly relevant anymore. What part is hard to change in microservice style architecture? The answer is not the inner components of an application anymore. The difficult things to change are some of the decisions that have been made about the microservices, for example, the ways they are integrated into the system, or the communication protocols between the involved applications and etc.
Thus, we at otto.de are drawing a difference between a micro-architecture of an application and the macro-architecture of the system. The micro-architecture is all about the internals of a vertical or a microservice, and is left completely in the hands of its respective team.
However, it is worth it to draw some guidelines for the macro-architecture, which can be defined by the interactions between the services. At otto.de, they are the following:
- Vertical decomposition: The system is cut into multiple verticals that belong entirely to a specific team. Communication between the verticals has to be done in the background, not during the execution of particular user requests.
- RESTful architecture: The communication and integration between the different services takes place exclusively via REST.
- Shared nothing architecture: There is no shared mutable state through which the services exchange information or share data. There are no HTTP sessions, no central data storage, as well as no shared code. However, multiple instances of a service may share a database.
- Data governance: For each data point, there is a single system in charge, a single “Truth”. Other systems have read-only access to the data provider via a REST API, and copy the required data to their own storage
Our architecture has begun going through a similar development cycle like a lot of other architectures out there. At the moment, we are standardizing the manner in which the microservices will be used at OTTO.
So far, I have elaborated a lot on the extent to which a system can be divided. However, the user is ultimately at the heart of all our development, and we want our software to be consistent, feeling all of a piece.
Therefore, the question becomes: in which way can we integrate a distributed system, so that the customers do not realize the distributed nature of our architecture?
The easiest kind of web-frontend integration of services is the use of hyperlinks.
Services are responsible for different pages, and the navigation takes place via the links between these pages.
Ajax is a suitable technology for less important or not visible areas of the site, which allows us to assemble pages from the different services.
The dependencies between the involved services are quite small. They mostly need to keep an agreement between each other on the used URLs and media types.
The deployment and the version management of these shared resources in a vertically sliced system is a whole different topic, which could become its own separate article. Without going into too much detail, the fact that the deployment of services needs to be independent while their assets need to be shared creates certain challenges of its own.
One of less known methods to integrate components of the site that come from different services is called Server- or Edge-Side Includes (SSI or ESI). It does not matter if we are dealing with Varnish, Squid, Apache or Nginx – the major Web-Server or Reverse Proxies support these includes.
The technique is easy: a service inserts an include statement with a URL in the response, and the URL is then resolved by the web server or reverse proxy. The proxy follows the URL, receives a response, and inserts the body of the response into the page instead of the include.
In our shop, for example, every page includes the navigation from the Search & Navigation (SAN) service in the following way:
<html> ... <esi:include src=“/san/...“ /> ... </html>
The reverse proxy (in our case Varnish) parses the page and resolves the URL of the include statements on that page. SAN then supplies the HTML-Fragment:
The Varnish proxy replaces the include within the HTML fragment and delivers the page to the user:
In this manner, the pages are assembled from the fragments of different services in a way that is completely invisible to the user.
The above-mentioned techniques prepare us for the Frontend-Integration. In order for services to successfully fulfill their requirements, some further problems need to be solved: services needing common data, while they must not share a database between each other. In our e-commerce shop, for example, several services need to process product data in a particular way.
A solution is the replication of data. All services, that, for example, need product data, regularly ask the responsible vertical (Product) for the necessary data, such that any changes in data can be quickly detected.
We are also not using any message queues that “push” data to their clients. Instead, services “poll” an Atom Feed whenever they need data updates and are able to process them.
In the end, we need to be able to deal with temporary inconsistencies – something that can, in any case, only be avoided at the cost of the availability of the services in a distributed system like ours.
No Remote Service Calls
Theoretically, we could avoid the replication of data in some cases, so that our services can synchronously (meaning, being inside the duration limits of a single request) access other services when needed. A shopping basket could also live without the redundant saving of the Product data and instead it could directly ask the product vertical for data when the shopping basket is presented on the page.
We avoid this for the following reasons:
- The testability suffers, when you are dependent on a different system for major features
- A slower service can affect the availability of the entire shop via a snowball effect
- The scalability of the system is limited
- Independent deployments of services are difficult
We have had great experience so far with the separation of verticals. It definitely helps, at least during the earlier stages, to learn to stick to a strict separation, Using vertical decomposition, services can be developed, tested and shipped independently of one another.
After three years of working on a system designed in the way outlined above, we have had, all in all, a very good experience. The biggest compliment for a concept is, of course, when someone else finds it interesting, or decides to copy it.
Looking back, if I were to do something differently, it would be to break up the monolith into even smaller pieces earlier in the process. The nearest future of otto.de definitely belongs to microservices in a vertical-style Architecture.
Many Thanks to…
Anastasia for translating this text.