Why Is Microservices Orchestration Becoming More Important in Modern Tech Stack

The microservice architecture enables to break of the single application into a bunch of small, loosely coupled services that have their own process. This architecture provides some benefits like improved scalability, better fault isolation, increased resiliency, etc.

Each microservice is independent, making it easy to scale up a single service rather than scaling the entire application.

Independent teams can focus on a single business process without the need to understand the big picture. Different teams can use different programming languages and frameworks of their choice to work their microservices. These microservices communicate with each other through networks.

You can check our comprehensive guide about monolithic vs. microservices architecture to see the benefits and drawbacks of microservices architecture in more detail.

The microservice architecture allows organizations to deliver large-scale applications rapidly. However, these small services must be able to communicate and work together to develop the larger application because the interaction among microservices is what builds the business capabilities of that application[1].

When building any business capability, you have to connect one or more microservices and other systems.

You need to have a clear business scope and clear business requirements. You cannot simply create services by randomly connecting them. Therefore, the critical question arises in a microservice architecture:

How do microservices collaborate and interact with each other?

When you start to model more complex logic, you have to deal with the problem of managing business processes and how your microservices communicate.

Today, you will hear loud symphonic music and watch a little dance to get answers: orchestration and choreography.

What a day full of art!

In computer science, some terms can be used with different meanings in different contexts. Therefore, to avoid misunderstanding, let’s define these terms in the context of this article by taking reference from Bernd Ruecker’s book :

• Command-driven communication: orchestration

• Event-driven communication: choreography

With orchestration, you have a central brain that guides and drives the process, like a conductor in an orchestra. Simply you have a component that coordinates one or more other components. This means the central component sends commands to other components.

With choreography, your services keep informed about its job and let them work out their details; like dancers, they have their own figures through the flow of choreography.

While reading this article, you will listen to music more than watching dance. In other words, the main focus of this article is microservices orchestration.

Before starting to read this article, it’s beneficial to become familiar with some terms and concepts:

Bounded Context

Remember Unix's philosophy: “Do One Thing and Do it Well.”

This philosophy begs the important question: How do you decide the boundaries of your service?

You developed monolith software, and dependencies grew to the point where they became unmaintainable (aka big ball of mud). A small change can lead to unpredictable side effects in other parts of the system.

In your big system, changes become more risky and expensive to deploy. You are considering switching to microservice architecture, but you do not know how to do it. Domain-Driven Design (DDD) mentality is here for you to solve this problem.

One key aspect of the DDD is the focus on the domain. The company’s business value is in its domain; by focusing on it, we are guaranteeing we are modeling what really matters or what differentiates the business[2]. The main idea of DDD is a common language shared by stakeholders, in particular domain experts and software developers, to reduce the risk of misunderstanding by reducing unclear requirements and business rules.

In this way, you create independent problem areas as Bounded Context, and each of them correlates to a microservice. Bounded contexts are relevant to the subdomain. Each subdomain has its own requirements and duties and may itself be subdivided. This division process repeats until the subdomain models are granular and actionable and can be translated into small and independent services by the implementing teams. Bounded contexts are established around these subdomains and form the basis for the creation of microservices[3].

What is the benefit of bounded context?

If you align your bounded contexts with business requirements, your microservices will be more loosely coupled. That provides autonomy to the team to design and implement solutions for their own specific business needs.

Thanks to autonomy, each term can have its own meaning in different bounded contexts and its own domain-focused solution. This natural segregation by bounded contexts enables the domains to change along with the business without affecting other domains[4].

Therefore, DDD can help you define your service boundaries with the support of bounded context, and it provides more autonomy to your team to design and develop their own microservices.

Workflow

A workflow is a particular set of actions that compose a business process.

Workflows commonly require multiple microservices, each with its own bounded context, performing its tasks and emitting new events to downstream consumers[5].

The workflow concept is important to understand microservices orchestration, one of the workflow patterns. The main concern in orchestration and choreography is to solve the problem of how multiple microservices can work together to fulfill larger business workflows.

Transaction

A transaction is a series of actions that must execute as a single unit. Even if one of the operations fails, the entire steps must be rolled back to leave the application in the previous stable state.

Command vs. Event

An event is something that happened, a fact.

For example, microservice A emits the ‘AIsCompleted’ to let other subscribed microservices know, but it has no expectations about what needs to happen based on this event. Other microservices might or not decide to react to the ‘AIsCompleted’ event.

On the contrary, if microservice A sends the command to microservice B, that means microservice A wants microservice B to do something. The intention of microservice A is clear, and microservice B cannot simply ignore this command.

Bern Reukter gives clear examples of the difference between command and event in his book:

If you tweet that you just got hungry, this is an event. It is broadcasted to the world, and it might lead to some action, maybe even having a real impact, such as a follower bringing some food to you (if you’re luckier than me). But more likely, it is totally ignored, and maybe not even read by anyone. This is OK for an event. The situation is different for a command. Imagine that you send an email to your favorite local restaurant to order a food delivery. Now you have a clear intent: you expect them to prepare and deliver your food. You would not use a tweet to order something at a restaurant.

It is important to mention that the difference between event and command is independent of communication type. In Twitter and mail, there is asynchronous communication. However, expectations are different for each one.

The intention of command is also different from the event in synchronous communication. If you pick up the phone and call someone (synchronous communication), you can say, “Hey, I’m hungry” (event) or “Hello, I want to order something” (command). The type of content is independent of the communication channel[6].

The difference between command and event helps you to understand microservice orchestration(command-based) by a small comparison with choreography.

What Is Microservices Orchestration?

Service Orchestration implements the business logic of a microservice by invoking and integrating one or more microservices and systems[7].

The central service sends commands to other microservices and provides coordination and integration between microservices. This central service is called an orchestrator in this article. It contains the entire business workflow logic for a given business process and sends specific commands to downstream microservices to tell them what to do.

The central service sends commands to other downstream services, awaits responses from the instructed microservices to ensure everything happens right, and handles the results according to workflow logic.

How Microservices Orchestration Works

Let’s imagine you want to build a new business capability as Order Service to provide your clients to buy a nice dress from your online shopping platform.

It requires the integration of several existing microservices, such as

• Inventory service

• Pricing service

• Shipping service

• etc.

You can build the business logic of Order Service so that it invokes Inventory, Pricing, and Shipping Services with the required message and finally sends the response back to the consumer of Order Service. The entire business workflow logic is self-contained within Order Service’s scope.

Ideal World Steps:

• Your client clicks the ‘Buy’ button

• Client order is received from Order Service

• Order Service sends a command to Inventory Service to change the beautiful dress stock and awaits the response about whether the stock is successfully changed

• Order Service receives a success message from Inventory Service

• Order Service sends the command to Pricing Service to charge and then awaits its response

• Order Service receives a success message from Pricing Service

• Order Service sends the command to Shipping Service to charge and then awaits its response

• Order Service receives a success message from Shipping Service

• You can see on your screen this message: Your order successfully is received

In this scenario, Order Service, the orchestrator, is responsible for keeping your orchestra in sync and coordinating the members to produce a cohesive musical piece during the performance.

Doesn't the music sound better with this cohesion?

The basic idea here is that loosely coupled microservices get their commands from the orchestrator, Order Service. It sends a call to each service and awaits a reply before proceeding. This central service acts like a brain, driving the whole execution process. It manages the steps of processes by instructing the other services to start new steps and triggering needed actions. It works much like a supervisor that oversees the whole process and delegates the actions to subordinate services[8].

Order Service keeps track of which parts of the workflow have been completed, which are in process, and which have yet to be started.

If any of the services fail to process, the service makes the Order Service know about that failure. Order Service would handle that failure and trigger any corresponding compensating actions (to undo change).

However, Order Service needs to know if the process has completely succeeded or if it has completely failed. It has no idea internal logic of other services, like how they handle commands.

Orchestrator is not interested in how musicians play their instruments.

The main concern of orchestrators is proving harmony by managing musicians. It stands in front of the orchestra, starting and stopping the music and controlling the quality of the music(fast, slow, loud, soft, etc.). S/he is not specifically interested in how musicians can play their instruments. Orchestrator doesn't need to know musicians’ internal way of playing. How to play the instrument is the responsibility of musicians, not the orchestrator.

Similarly, in microservice orchestration, you must ensure that the orchestrator’s bounded context is limited strictly to workflow logic and contains minimal business fulfillment logic[9]. The orchestrator is responsible only for orchestration and workflow logic. It is not responsible for fulfilling the business logic of any of the microservices themselves.

Myth: Commands Require Synchronous Communication

The downstream services can use either synchronous or asynchronous communication. These downstream services may call several other downstream services too. This style of communication is known as service chaining. From the perspective of the orchestrator, such as Order Service, the existence of multiple chained downstream services is irrelevant[10].

However, there is the myth that commands require you to communicate synchronously, leading to temporal coupling.

As mentioned before, the choice between command and event is independent of choosing asynchronous or synchronous communication. Temporal coupling results from synchronous communication alone, not from choosing to use commands.

Microservices Orchestration can be implemented as Event-Driven Orchestration and Direct-Call Orchestration, depending on synchronous or asynchronous communication between the orchestrator and other services.

In both models, the orchestrator sends commands to microservices, and all downstream microservices communicate with just the orchestrator microservice.

The orchestrator is still responsible for workflow logic in both. The only thing changing is the communication between the orchestrator and downstream microservices.

Let’s look closer at both designs.

Event-Driven Orchestration (Asynchronous)

In this pattern, The orchestrator keeps a materialization of the events issued to services A, B, and C and updates its internal state based on the results returned from the worker microservice[11].

It keeps different stages of the workflow. Depending on the response of downstream microservices, the orchestrator can make decisions and select the next step according to the workflow logic.

One of the advantages of this pattern is when the orchestrator or other downstream microservices goes down, their messages will be stored in the queue. Therefore, when problems that lead to a down microservice are solved, messages will be ready to be consumed by the consumer.

Direct-Call Orchestration (Synchronous)

Orchestration can also use a request-response pattern in communication between the orchestrator and downstream services. Orchestrator synchronously sends a request to microservice and awaits a response. In every request, the orchestrator blocks while awaiting a response. Once it gets the response, the orchestrator updates its internal state and calls other services.

Depending on which one you use between these two designs, the orchestrator can keep track of event streams or response-request results in the workflow. However, you must be careful to create a ‘God’ service with much of a central governing authority.

Orchestrated Transactions

You can revert transactions from any point in the workflow by modeling transactions as orchestrated. If one service cannot complete the transaction, you can roll back all transactions by reversing the workflow logic and ensuring each microservice successfully reverses actions.

You can monitor the progress and state of the transaction closely owing to the centralized nature of orchestration.

The transaction can be canceled at any workflow point because of any error message from downstream microservices.

The Orchestrator can use either synchronous or synchronous calls. The critical point here is that, in the first place, the downstream service will try to perform necessary operations according to its internal logic. After its own error handling logic, if an error still exists, it returns a failure response to the orchestrator.

Orchestrator must now maintain its rollback logic based on the current state of that response by issuing rollback commands to all required microservices. This means the orchestrator service send command to other required services to make them roll back their last state.

However, if there were fail to rollback, the orchestrator service decides what to do next. As mentioned before, while each downstream microservices are responsible for its own retry policy, error handling, and failure management, the orchestrator is only responsible for workflow management. This is why the orchestrator should manage any failure situation by deciding the next step in the workflow.

Ordinarily, if the transaction has been rolled back, it is also the responsibility of the orchestrator to decide what to do next to finalize the processing of that event. There are some options: retry the event number of times, discard it, terminate the application, or output a failure event[12]. Therefore, the orchestrator is responsible for issuing rollback commands and awaiting microservices' confirmation. How the orchestrator finalizes the process is up to you by choosing what kind of stream you design in your application.

The orchestrated design gives you better visibility into workflow dependencies, more flexibility for changes, and clearer monitoring options compared to choreography design[13]. They can handle more complicated distributed transactions thanks to their centrist design. If your workflow changes and your application includes many independent microservices, it may be beneficial for you to consider orchestration design.

Benefits of the Microservice Orchestration

Simpler Dependencies

The orchestrator invokes the downstream microservices. However, these microservices don’t invoke the orchestrator. In that picture, the orchestrator depends on the downstream microservices, not vice versa. Therefore, there are no cyclic dependencies.

Less Coupling

If you design each service by implementing an API invoked by the orchestrator, it does not need to know other services.

Improvement of the Separation of Concerns & Simplified Business Logic

As we mentioned, the orchestrator has coordination logic in its local. The domain objects are simpler and do not know the long-running action they participate in.

Reliability

It enables you to coordinate the fulfillment process if you design your orchestrator to know what services are needed to carry out the operation and to enable you to decide when to send commands to those services. Orchestrator decides what to do if there is any failure during the process. Therefore, fault tolerance scenarios are much simpler than choreography, which has point-to-point communications.

Scalability

When adding a new service into orchestration, only the orchestrator needs to modify the interaction rules, while in choreography, all the interacting services need to be modified.

Easy to Understand

This approach allows you to understand how this process is supposed to work. This can make onboarding new people easier and help them to understand the core part of the system better. You have less work to monitor and observe your application comparing orchestration.

Drawbacks of the Orchestration Approach

‘Creating God’

The relationship between the orchestrator and the downstream services must be explicitly defined because orchestration can leads to a tight coupling between services. You must be sure whether orchestration is responsible only for managing business workflow. You don't want to create a “God” service that gives commands to other dump services. There is a high risk of causing poor encapsulation and ill-defined bounded context. This is the big enemy of microservices' loosely coupled nature in a perfect world.

You cannot scale your microservices through your diversified teams if there is tight coupling. This will not be what you want. Therefore, your orchestrator should not contain business logic as much as possible and delegate full responsibility for business logic to downstream services. Each downstream service must be strong enough to process its own business logic.

This approach can lead to your services having little behavior, just taking orders from the orchestrator. It is important to consider that your microservices congregate orchestrated flows together and have their own local state and internal logic. They are in charge of their own state and behavior.

Low Performance

Downstream microservices talk via a centralized orchestrator; therefore, latency is higher compared with choreography. There is no direct communication between downstream microservices. They communicate with the orchestrator, and the orchestrator manages the steps of processes by instructing downstream services. Giving instructions and awaiting a response increase the latency. In addition, the throughput is bound to the capacity of the orchestrator.

Single Point of Failure

If the orchestrator goes down, no services can talk to each other. To mitigate this, the orchestrator must be highly available.

Microservices Orchestration vs. Microservices Choreography

To make the comparison, it will be logical to discuss choreography briefly.

With choreography, you inform each part of the system of its job and let it work out the details, like dancers all finding their way and reacting to others around them in a ballet[14]. They know their business goals and rely on certain events or responses from other services that determine how they function. They are just interested in certain responses or events to take appropriate actions. Instead of being controlled by an orchestrator, microservices interact directly with each other.

For example, no one service knows about other services when you think about event-driven architecture with choreography design. They only need to know what to do when a certain event is received. Parties would subscribe to a specific topic that refers to a certain type of event and is not interested in where these events came from. They are only interested in when a certain event is received. Because the process implementation is decomposed, it makes for much less coupled architecture. In that approach, you avoid the concerns about centralized logic.

Even if the choreographed approach is significantly more decoupled, monitoring and tracking can be more difficult and complex.

In that design, microservices are not managed by a central service. They know their goals and rely on other services' responses or events that determine how they function. If you implement this design in event-driven architecture, each service publishes messages, and other services subscribe and listen to events they are interested in to take the appropriate actions. This approach can be more loosely coupled, flexible, and amenable to chance.

On the other hand, the orchestrator sends a command message to downstream microservices, telling it what operation to perform. After this microservice has performed the operation, it sends a reply message to the orchestrator. The orchestrator then processes the message and determines which next step to perform.

Microservice orchestration can provide better workflow visibility depending on the case because of its centralist approach. Even if orchestration provides better visibility into workflow dependencies, more flexibility for changes, and more precise monitoring options, there is a risk orchestrator becomes ‘God.’

Sam Newman mentions this in his book: ‘In general, I have found that systems that tend more toward the choreographed approach are more loosely coupled, and are more flexible and amenable to change. You do need to do extra work to monitor and track the processes across system boundaries, however. I have found most heavily orchestrated implementations to be extremely brittle, with a higher cost of change. With that in mind, I strongly prefer a choreographed system, where each service is smart enough to understand its role in the whole dance.’

Even though he accepts you need to do extra work for the monitoring system, he suggests you build a monitoring system to track what each service does as independent entities.

However, it will be good to mention that you can find both communication styles, orchestration, and choreography in the same architecture.

The Bottom Line: Importance of Microservices Orchestration

We have lots of microservices in our architecture.

They have their own business, domain, and logic. They look good when they are alone. However, they need to serve a more important purpose.

To do that, they must communicate through the network with the specific design.

Orchestration provides you to put a service in charge of the other services. In this way, the orchestrator is aware of the entire workflow. If there is any problem in the overall process, it will know and take some actions through that. While the orchestrator manages the process execution, it will be easier for support to fix problems and manage the entire system’s development.

It can be beneficial when the business use case requires one service to handle all the interactions with other services and systems. It is usually suitable for interactive services.