Reactive Microservices

TL;DR: In the modern world of software development, we need to find techniques that allow us to isolate the inherent complexity, not just of our software, but of our development processes as well. This is where microservices enter the picture.

In the early days of software development, a single developer, or a small group, would build an application to support a small number of users. In this environment, building a monolithic style application made sense. It was the simplest thing we could do. However, over the years, software complexity has grown. Today, multiple teams are working with many different technologies, supporting millions of users. In this environment, the monolithic application becomes a burden, rather than a benefit.

The Software Spectrum

We often talk about apples and pears terms: you're either building monoliths or you're building microservices. But that's not really fair because the reality is monoliths and microservices don't exist in kind of a apples and pears situation. They exist more on a spectrum with monoliths on one end, microservices on the other. Most applications that get built, live somewhere in-between. The thing that we have to recognize is that monoliths and microservices both have advantages and disadvantages. No matter which one we pick we're going to be giving up one thing in order to gain something else. It's then a big part of our job to look at the differences between them and look at the benefits or disadvantages to each and try to balance them. In doing so, we place ourselves somewhere on that spectrum.

Monoliths

Our software spectrum has monoliths at one end and microservices at the other.

The worst case of a monolith is one that is called the ball of mud or spaghetti code. In this scenario, there is no clear isolation in the application. There are a lot of complex dependencies. And is really hard to understand and to modify.

To clean up the ball of mud we introduce isolation into the application:

  • We divide the application along clear domain boundaries.
  • We introduce libraries that help isolate related pieces of code.
  • Libraries provide a clean and consistent interface.

Characteristics of a Monolith:

  • Deployed as a single unit.
  • Single shared database.
  • Communicate with synchronous method calls.
  • Deep coupling between libraries and components (often through the DB).
  • “Big Bang” style releases (the world stops, everybody stops doing code changes, etc.).
  • Long cycle times (weeks or even months).
  • Teams carefully synchronize features and releases.

How do we scale a monolith?

  • Multiple copies of the monolith are deployed.
  • Each copy is independent. They don’t communicate with each other. They don’t know the others exists.
  • The database provides consistency between deployed instances.

Advantages of the monolith.

  • Easy Cross Module Refactoring.
  • Easier to maintain consistency.
  • Single deployment process.
  • Single thing to monitor.
  • Simple scalability model.

Disadvantages of the monolith.

  • Limited by the maximum size of a single physical machine.
  • Only scales as far as the database allows.
  • Components must be scaled as a group (you can have unnecessary use of resources).
  • Deep coupling leads to inflexibility.
  • Development is typically slow (change is difficult, build times long).
  • Serious failures on one component often bring down the whole monolith.
  • Redistribution of a load can cause cascading failures.

Microservices

Service Oriented Architecture (SOA)

One of the ways that you can improve the monoliths' situation rather than having a large ball of mud kind of scenario, is that you can introduce an additional isolation. You do that by separating your monolith along clear domain boundaries usually in the form of different libraries for different areas of the domain. There's another way that we can introduce additional isolation and that's using something called Service Oriented Architecture.

  • Services don’t share a database.
  • All access must go through the API exposed by the service.
  • Services may live in the same process (monolith) or may be separated (microservices).
  • If one service needs access to the data from another service, it can request the data by making an API call to the service that owns the data. 

Microservices are a subset of Service Oriented Architecture. The difference between them is Service Oriented Architecture doesn't dictate any requirements around deployment. You can deploy Service Oriented Architectures as a monolith or you can deploy each of your services independently. If you deploy them independently that's when you start to build microservices.

Micro services require that each of those services are independently deployed. This means that they can be put on to many different machines. You can have any number of copies of individual services depending on your requirements. We still keep all of the rules around Service Oriented Architecture: maintaining our own datastore, ensuring that our services communicate only through a clearly defined API, all of those things still apply. We've just added that extra requirement, that we have to deploy the individual services independently. By doing this it means that microservices are independent and self-governing. 

Microservice Characteristics

  • Each service is deployed independently.
  • Multiple independent databases.
  • Communication is synchronous or asynchronous (Reactive Microservices).
  • Loose coupling between components.
  • Rapid deployments (possibly continuous).
  • Teams release features when they are ready.
  • Teams often organized around a DevOps approach.

Scaling a Microservice Application

  • Each microservice is scaled independently.
  • Could be one or more copies of a service per machine.
  • Each machine hosts a subset of the entire system.

Advantages of the Microservice System

  • Individual services can be deployed/scaled as needed.
  • Increased availability. Serious failures are isolated to a single service.
  • Isolation/decoupling provides more flexibility to evolve within a module.
  • Supports multiple platforms and languages.

Microservice Team Organization

  • Microservice often comes with organizational change.
  • Teams operate more independently.
  • Release cycles are shorter.
  • Cross team coordination becomes less necessary.
  • These changes can facilitate an increase in productivity.

Disadvantages of the Microservice System

  • May require multiple complex deployment and monitoring approaches.
  • Cross service refactoring is more challenging.
  • Requires support for older API versions.
  • Organizational change to microservices may be challenging.

Responsibilities of a Microservice

How big a microservice should be?

Single Responsibility Principle

  • The SRP applies to all object oriented classes, but works for microservices as well.
  • A microservice should have a single responsibility (e.g. managing accounts).
  • A change to the internals of one microservice should not necessitate a change to another microservice.

How do we decide where to draw those lines? How do we decide when to build our microservices and what the proper responsibilities are?

  • Bounded Contexts are a good place to start building microservices.
  • They define a context in which a specific model applies.
  • Further subdivision of the Bounded Context is possible.

Principles Of Isolation

When you want to answer the question of how big should a microservice be, you're really asking the wrong question. The right question is more about how can I isolate my microservices? If you can figure out how to isolate them appropriately, then you'll probably find a way to make them a good size.

As we move from Monoliths to Microservices we are introducing more isolation. Isolation provides reduced coupling and increased scalability.
Reactive Microservices are isolated in:

  • State
  • Space
  • Time
  • Failure

Isolation Of State

  • All access to a Microservice’s state must go through its API.
  • No backdoor access via the database.
  • Allows the microservice to evolve internally without affecting the outside.

Isolation In Space

  • Microservices should not care where other microservices are deployed.
  • It should be possible to move a microservice to another machine, possibly in a different data center without issue.
  • Allows the microservice to be scaled up/down to meet demand.

Isolation In Time

  • Microservices should not wait for each other. Requests are asynchronous and non-blocking.
  • More efficient use of resources. Resources can be freed immediately, rather than waiting for a request to finish.
  • Between microservices, we expect eventual consistency.
  • Provides increased scalability. Total consistency requires central coordination which limits scalability.

Isolation Of Failure

  • Reactive Microservices also isolate failures.
  • A failure in one microservice should not cause another to fail.
  • Allows the system to remain operational in spite of failure.

Isolation Techniques

Bulkheading

Bulkheading is a tool used to isolate failure. Failures are isolate to failure zones. Failures in one service will not propagate to other services. Overall systems can remain operational (possibly in a degraded state).

Bulkheading is a term that comes from shipbuilding. Bulkheads and ships are used to create separate watertight compartments in the hull of the ship. This means that a failure in the hull will potentially flood one compartment but the others are going to remain safe. And as a result the ship won't sink, this allows the ship to be resilient in the face of failure. Bulk heading in terms of software is a similar kind of concept. What we try to do is create failure zones within our application so that, when a failure occurs, it's isolated within that zone and it's not going to propagate to other services. 

Netflix is a really good example of this. The "recently watched" or "my list" features, when they go down, that's due to a microservice being down. What Netflix does in that case is it simply hides those features. The rest of Netflix can continue to operate. You can continue to watch your movies or TV shows or whatever. You just can't access the more personalized aspects like "recently watched" or "my list".

Circuit Breakers

What happens when a service depends on another that is overloaded?

  • Calls to the overloaded service may fail.
  • The caller may not realize the service is under stress and may retry.
  • The retry makes the load worse.
  • Callers must be careful to avoid this.

Circuit Breakers are a way to void overloading a service. They quarantine a failing service so it can fail fast. This gives the failing service time to recover without overloading it.

A circuit breaker has 3 states: Close (Normal), Open (Fail Fast), and Half Open.

All the external service calls must pass through the circuit breaker. It inits in the Close state, if the request fails, the circuit breaker becomes open so all the following calls are not going to be made. After a timeout or a reset function, the circuit breaker becomes half open, it will try the following service call, if it fails, it returns to the Open state, if it succeeds it will pass to the closed state as normal.

Message Driven Architecture

Async, non-blocking messaging allows us to decouple both time and failure. Our services are less likely to fail. They become less brittle and they're not dependent on things to happen immediately. They allow for some time to happen and that's a benefit. It results in systems that are going to be more robust.

  • Services are not dependent on the response of each other.
  • If a request to a service fails, the failure won’t propagate.
  • The client service isn’t waiting for a response. It can continue to operate normally.

A Reactive System is built on a foundation of Asynchronous, non-blocking messages.

Autonomy

We build systems using the principles of isolation to achieve what we call autonomy. Autonomy is basically the idea that each of our services can operate independent of another. 

Autonomous Services

  • Microservices can only guarantee their own behavior (via their API).
  • Isolation allows a service to operate independent of other services.
  • Each service can be autonomous.
  • Autonomous services have enough information to resolve conflicts and repair failures.
  • They don’t require other services to be operational all the time.

Benefits of autonomous services

  • Autonomy allows for stronger scalability and availability.
  • Fully autonomous services can be scaled indefinitely.
  • Operating independently means that they can tolerate any amount of failure.
  • Few services will be fully autonomous, but the closer we get, the better.

Achieving Autonomy

  • Communicate only through asynchronous messages.
  • Maintain enough internal state for the microservice to function in isolation.
  • Use eventual consistency.
  • Avoid direct, synchronous dependencies on external services.

Gateway Services

Managing API complexity

  • Microservices can lead to complexities in the API.
  • A single request may require information from multiple microservices.
  • Clients could send out many requests, and aggregate the results, but this may be too complex.
  • How can we manage complex APIs that access many microservices?

Those API complexities can be solved by using API Gateway Services

  • Requests can be sent through a Gateway Service.
  • A Gateway service sends the requests to individual microservices and aggregates the response.
  • Logic for aggregation is moved out of the client and into the Gateway Service.
  • Gateway handles failures from each service. Client only needs to deal with possible failures from the Gateway.
  • We can have domain specific gateways.