The microservices architecture has been getting a lot of attention recently as software teams are looking for new ways or improving their release workflows. Amazon, Netflix and Ebay are among the companies that are openly embracing this way of building software and they've contributed back to the community by publishing their own experience and developing tools that can help others to adopt.

There isn't a strict definition for microservices but the guiding principle is to build an application by splitting its business components in small services that can be deployed and operated independently from each other. They interact with each other over via simple protocols like REST to complete actions but they have no knowledge of how other services work internally. From a user perspective, everything looks like a single application running that they can interact with via the user interface or its API. But developers can now organize in smaller teams specializing in different services, with different stacks and decoupled deployments.

In this introduction, we will see what are the core guiding principles behind the microservice architecture and we'll look at the benefits and drawbacks of such approach.

Monolithic applications vs. microservices

Looking at monolithic applications is an easy way to explain the basic idea behind microservices and why you may want to adopt this kind of architecture. In a monolithic web application, all the logic is generally contained in a single server-side application that interacts with a database to retrieve and persist data. Users interact with the application via their browser, mobile device or an API but the core of the application is built as a single module.

Let's illustrate that with an example – we're building a new online pizza delivery application that we'll affectionately call Pizzup. Let's skip a lot of product development steps for now to have a look at what our final system may look like. Our customers should be able to see the list of pizza available, select which one they want to get, pay and get their bill. We also want them to be able to create an account to make it easier to order again next time. And because we're cool we want them to be able to track their delivery in real-time. Of course, we'll provide all these capabilities to both desktop and mobile devices.

In a monolithic system, our application would look more or less like the diagram below.

monolithic system

There are different modules in the backend that are in charge of specific business needs (account management, billing, payments, etc). All the modules are in the same codebase and they're packaged and deployed as a single application. This could be easily done with frameworks like Django for Python, Ruby-on-rails for Ruby, or Laravel for PHP. Your UI could be served by the same application server and be responsive to adapt to mobile devices.

There are some advantages to building a system like that. First of all, it's easy to deploy since you only have one application server (and a database) to manage. It's also easy to do end-to-end testing as all modules run in the same process and most of the communication between them is happening at a language level. On top of that, there's a single codebase to manage which makes it easier for rapid iteration at the beginning of a project.

But a lot of these advantages start to erode as a project grows in size. It may have been great at the beginning of the project to have a tight coupling between the business components. But what happens when we decide to refactor the billing system to cater for different types of postcodes for instance? Even if we managed to keep the architecture clean we have the risks of breaking the application every time we publish new changes to the main repository. All the modules live in the same application so if Joe breaks the build while updating the billing system then Sarah who was working on the improvements to the inventory is also blocked. She might also have to switch context to handle some merges that are not relevant to her work.

Deploying is also an issue in a large monolith application. When Sarah is done with her improvement she will have to test not only that the inventory system works properly, but she'll also have to check all the other modules that are part of the system. And the tiniest issue to any of the modules will stop all the others from going to production. Once again this is something that is going to slow down the development cycle and increase risks for the next release as more and more changes will bulk up while teams are coordinating for the deployment.

Another aspect of a monolithic system that can be hard to deal with as it grows is scaling. In our Pizzup application, we have many different modules that are here to support different types of business needs. They coexist on the same application server as if they all contribute equally but it's quite likely that some modules will require more resources than others. Our inventory system is probably not going to cause us much grief, but the live-tracking module is probably going to put some load on our system as our orders grow in number. Unfortunately, in a monolith you can't optimise per module and you have to scale everything together. That means that if we fail to scale the live-tracking module properly then all the system might fail if there's an excessive load on that module. It would be much better to be able to isolate the live-tracking failure so that our customers can keep on placing orders.

This is why microservices are becoming popular today. In a microservice architecture, each business capability can become its own service that can be deployed independently from the other. From a customer perspective, it still looks like a single application and they can either access the application via a frontend server or a mobile application that interacts with all the different microservices via an API gateway.

The microservice equivalent of our monolithic Pizzup system would look like the diagram below.

Pizzup Microservices System New

In this example, we choose to split it according to the modules we identified earlier. Each module has become its own small service. There is an API gateway that simplifies the communication between the user devices and the application. Instead of having to know all the microservices that exist in the system the Web UI and mobile client only needs to know the public APIs available. The API gateway will then dispatch the calls appropriately and compose the responses as needed.

A key advantage of such architecture is that it makes it easier to grow and extend. The team working on the refactoring of the billing service can do so without being scared of slowing down other teams. If they break their build it will only affect the development of the billing service. The team working on the inventory service won't know of it and can focus on the improvements they need to deliver.

This also means that deployment is easier and faster. As soon as the team on the inventory service is done they can deploy their changes to production once it passes the test suite for the inventory service. Having other microservices broken in their own development environments does not prevent them from releasing. Of course they'll have to run some end-to-end or contract tests to make sure that the entire system works as expected but their deployment is self contained.

With regards to scaling you can start doing some more interesting things. You can start allocating different resources for different services, but you can also start adopting different technologies to use what is best in every situation. This is not to encourage you to have each service written in a different language but if you find out that one service could be much more performant in a different technology then you are able to rewrite that service without having to rewrite your entire application.

A word of caution as it's not all easy-peasy in the world of microservices. You will gain a lot of flexibility but you'll have to deal with the complexity of having a distributed system. You'll need to think about partial failures, testing strategies, monitoring, data storage. In the next section, we'll go into the main steps to transition to a micro service architecture as well as the key things to watch out for.

Building microservices

Start with a monolith

The first of rule of building microservices is that you shouldn't start with it for a green field project. If you don't have any users for your application chances are that the business requirements are going to rapidly change while you're building your MVP. This is simply due to the nature of software development and the feedback cycle that needs to happen while you're identifying the key business capabilities that your system needs to provide. For this reason, it is much easier to keep all the code and logic within a single codebase as it makes it easier to move the boundaries of the different modules of your application.

For instance with Pizzup we might have started with a very basic idea of the problem we want to solve for our customers: we want people to be able to order pizza online.

User Monolith

As we start thinking of the pizza ordering issue we will begin to identify the different capabilities required in our application in order to fulfil that need. We'll need to be able to manage a list of the different pizzas we can make, we'll need to let customers pick one or many pizzas, handle the payment, schedule the delivery and so on. We may as well decide that letting our customers create an account will facilitate re-ordering the next time they use Pizzup, and after talking to our first users we might figure out that live-tracking of the delivery and mobile support would definitely be giving us an advantage on the competition.

User Admin Monolith

What was a simple need at the beginning quickly turns into a list of capabilities that you need to provide.

Microservices work well when you have a good grasp of the roles of the different services required by your system. They're much more difficult to handle if the core requirements of an application are still being worked out. It's indeed quite costly to redefine service interactions, APIs and data structures in microservices as you may have many more moving parts that need to be coordinated. This is why our advice is to keep things simple until you have collected enough user feedback to give you confidence that the basic needs of your customers are understood and planned for.

A word of caution though as building a monolith can quickly lead to complicated code that will be hard to break down in smaller pieces. Try as much as you can to have clear modules identified so that you can extract them later out of the monolith. You can also start by separating the logic from your web UI and make sure that it interacts with your backend via a RESTful API over HTTP. This will make the transition to microservices easier in the future when you start moving some of the API resources to different services.

Organize your teams the right way

Up until now it would have seemed that building microservices is mostly a technical affair. You'll need to split a codebase in multiple services, implement the right patterns to fail gracefully and recover from network issues, deal with data consistency, monitor service load, etc. There will be a bunch of new concepts to grasp but one thing that must not be ignored is that you'll need to restructure the way your teams are organized. 

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

Conway's Law

Conway's Law is a real thing that can be observed in all types of teams, and if a software team is organized with a backend team, a frontend team and an ops team separated they will end up delivering separate frontend and backend monoliths that get thrown away to the ops team so that they can manage it in production.

This type of structure is not a good fit for microservices as each service can be seen as its own product that needs to be shipped independently of the others. Instead, you should create smaller teams that have all the competencies required to develop and maintain the services they're in charge of. Werner Vogels, CTO of Amazon, described this situation with the phrase "you build it, you run it". There are great benefits to arranging your teams this way. First of all your developers will get a better understanding of the impact that of their code in production – this will help produce better release and reduce the risk of seeing issues released to your customers. Secondly, your deployments will become a second nature for each team as they will be able to work together on improvements to the code as well as the automation of the deployment pipeline.

Splitting the monolith

When you've identified the boundaries of your services and when you've figured out how you can change your teams to be more vertical in terms of competencies you can start splitting your monolith. Here are the key points to think about at that time.

Keep communication between services simple

If you're not already using a RESTful API now would be a good time to adopt it in your system. As Martin Fowler explains it you want to have "smart endpoints and dumb pipes". This means that the communication protocol between your services should be as simple as possible, only in charge of transmitting data without transforming it. All the magic will happen in the endpoints themselves – they receive a request, process it, and emit a response in return.

This is also where microservices can be distinguished from SOA by avoiding the complexity of the Enterprise Service Bus. Microservice architectures strive to keep things as straightforward as possible to avoid tight coupling of the components. In some cases, you might find yourself use an event-driven architecture with asynchronous message-based communications. But once again you should look into basic message queue services like RabbitMQ and avoid adding complexity to the messages transmitted over the network.

Divide your data structure

It is quite common to have a single database for all the different capabilities in a monolith. When a user access it's order you'll look directly in the user table to display the customer information, and the same table might be used to populate the invoice managed by the billing system. This seems logic and simple but with microservices you will want the services to be decoupled so that invoices can still be accessed even if the ordering system is down and because it allows you to optimize or evolve the invoice table independent of others.  This means that each service might end up having its own datastore to persist the data that it needs.

It obviously introduces new problems as you will end up having some data duplicated in different databases. In this case, you should aim for eventual consistency and you can adopt an event-driven architecture to help syncing data across multiple services. For instance, your billing and delivery tracking services might be listening for events emitted by the account service when a customer update their personal information. Upon reception of the event those services will update their datastore accordingly. This event-driven architecture allows the account service logic to be kept simple as it doesn't need to know all the other dependent services. It simply tells the system what it did and other services listen and act accordingly.

You can also choose to keep all the customer information in the account service and only keep a foreign key reference in your billing and delivery service. They would then interact with the account service to get the relevant customer data when needed instead of duplicating existing records. There isn't a universal solutions for these problems and you'll have to look into each specific case to determine what the best approach is.

Build for failure

We've seen how microservices can provide you with great benefits over a monolithic architecture. They're smaller in size and specialized which makes them easy to understand. They're decoupled which means that you can refactor a service without having to fear breaking the other components of the system, or slowing down the development of the other teams. They also give more flexibility to your developers as they can pick different technologies if required without being constrained by the needs of other services.

In short, having a microservice architecture makes developing and maintaining each business capability easier. But things become more complicated when you look at all the services together and how they need to interact to complete actions. Your system is now distributed with multiple points of failure and you need to cater for that. You need to take into account not only cases where a service is not responding, but also be able to deal with slower network responses. Recovering from a failure can also be tricky at times as you need to make sure that services that get back up and running do not get flooded by pending messages.

As you start extracting capabilities out of your monolithic systems make sure that your designs are built for failure right from the beginning.

Monitoring vs testing

Testing is another drawback of microservices compared to a monolithic system. An application that is built as a single codebase doesn't need much to have a test environment up and running. In most cases, you'll have to start a backend server coupled with a database to be able to run your test suite.

In the world of microservices things are not as easy. When it comes to unit tests it will still be quite similar as the monolith and you shouldn't feel more pain at that level. However when it comes to integration and system testing things will become much more difficult. You might have to start several services together, have different datastores up & running, and your setup might need including message queues that you did not need with your monolith. In this situation, it becomes much more costly to run functional tests and the increasing number of moving parts makes it very difficult to predict the different types of failures that can happen.

This is why you'll need to put a great emphasis on monitoring to be able to identify issues early and be able to react accordingly. You'll need to understand the baselines of your different services and be able to react not only when they go down, but also when they're behaving unexpectedly. One advantage of adopting a microservice architecture is that your system should be resilient to partial failure, so if you start to see abnomalies in the delivery tracking service of our Pizzup application it won't be as bad as if it were a monolithic system. Our application should be designed so that all the other services respond properly and let our customers order pizzas while we restore the live-tracking.

Reduce deployment friction

Releasing a monolithic system to production manually is a tedious and risky effort but it can be done. Of course, we do not recommend this approach and encourage every software team to embrace continuous delivery for all types of development but at the beginning of a project you might do your first deployments yourself via the command line.

This approach is not sustainable when you have an increasing number of services that need to be deployed multiple times a day. So, as part of your transition to microservices it is critical that you embrace continuous delivery to reduce the risks of release failure, as well as making sure that your team is focused on building and running the application, rather than being stuck deploying it. Practicing continuous delivery will also mean that your service have passed acceptance tests before going to production – of course bugs will occur but over time you will build a robust test suite that should increase your the confidence of your team in the quality of the releases.

Running microservices is not a sprint

There's a general hype going on with microservices and it's somewhat justified. For complex projects, they offer a greater flexibility in the way you can build and deploy software, and they help identify and formalize the business components of your system which comes in handy when you have several teams working on the same application. But there's also some clear drawbacks to managing distributed systems and splitting a monolithic architecture should only be done when there's a clear understanding of the service boundaries.

The adoption of microservices should be seen as a journey rather than the immediate goal for a team. Start small to understand the technical requirements of a distributed system, how to fail gracefully and scale individual components. Then you can gradually extract more and more services as you gain experience and knowledge.

The microservice architecture is still fairly young but it's a promising way of developing applications and it's definitely worth looking into but remember that it might not (yet) be a good fit for your team.

Posted by Sten Pittet

8 min read

Products discussed
Bitbucket Pipelines logo
Bitbucket Pipelines: Built-in Continuous Delivery Solution