How to manage distributed transaction in Microservices?

In a Microservices architecture, services are designed to be small, autonomous, and loosely coupled. This design philosophy provides many benefits, including scalability, flexibility, and faster development cycles. 
However, managing transactions across multiple microservices can be a challenging task. In this article, we will explore how to manage distributed transactions in microservices and provide relevant code examples.
There are multiple ways to manage transactions in Microservices like applying SAGA Pattern, using 2 Phase commit or using Event Sourcing Design Pattern. In this article, we will understand all of those but before that, let's understand what is distributed transactions and what are challenges related to transaction management in Microservices. 

1. What are Distributed Transactions in Microservices?

A distributed transaction involves multiple services that need to work together to complete a transaction. 

In a Microservices architectureMicroservices architecture, each service may have its own database, and transactions that require data from multiple services may need to coordinate the work of all these services to ensure that the transaction is completed successfully.

For example, when a user makes a purchase, the transaction may involve several services, such as the order service, payment service, and inventory service.

Challenges of Managing Distributed Transactions in Microservices

Managing distributed transactions in a microservices architecture can be challenging due to the following reasons:

1. Complexity
Managing transactions across multiple services can be complex, and coordinating the work of all these services can be challenging.

2. Failure Handling
In a distributed system, failures are inevitable, and handling these failures in a distributed transaction can be challenging.

3. Scalability
Microservices architecture is designed to be scalable, and managing transactions across multiple services can be challenging in a highly scalable environment.

4. Latency
Coordinating the work of multiple services can introduce latency, which can impact the performance of the system.

6 Ways to Manage Distributed Transactions in Microservices?

There are several approaches to manage distributed transactions in a microservices architecture. Here are some of the most popular approaches:

1. Two-Phase Commit (2PC)

Two-phase commit is a protocol used to ensure that all participants in a distributed transaction agree to commit or abort the transaction. In this approach, a coordinator service is responsible for coordinating the work of all the services involved in the transaction. 

The coordinator asks all the services to prepare for the transaction, and if all the services are ready, the coordinator sends a commit message to all the services. If any of the services fail to prepare or respond, the coordinator sends an abort message to all the services.

Here is a nice diagram which shows how two phase commit works:

2. Saga Pattern

The Saga pattern is a pattern used to manage long-running transactions in a distributed system. In this pattern, each service involved in the transaction performs a local transaction and sends a message to the next service to perform its transaction. 

If any of the services fail, the Saga can be rolled back by sending a compensating transaction to undo the work that has already been done.

Here is a nice diagram which shows how Saga pattern works in a Microservices architecture:

3. Event-Driven Architecture

Event-driven architecture is an architectural pattern that involves the use of events to trigger actions in a system. In a distributed transaction, each service can publish events when it has completed its part of the transaction. 

Other services can then subscribe to these events and perform their part of the transaction. This is probably the simplest solution of managing distributed transactions in Microservices architecture. 

Here is a nice diagram to show how Event Drive Architecture works:

Code Example of SAGA and 2-Phase Commit in Java Microservices

Let's take a look at some code examples of how to manage distributed transactions in microservices using the two-phase commit and Saga pattern approaches.

• Two-Phase Commit (2PC)

Here is a sample code of two phase commit using Java:

@Service
public class OrderService {

    @Autowired
    private PaymentService paymentService;

    @Autowired
    private InventoryService inventoryService;

    @Transactional
    public void placeOrder(Order order) throws Exception {
        // Reserve inventory
        inventoryService.reserveInventory(order);

        // Charge payment
        paymentService.chargePayment(order);

        // Commit transaction
        // This is handled by the transaction manager
    }
}

Here is how 2-phase commit look like in a sequence diagram:

•  Saga Pattern

Now that we have seen code example of using 2 phase commit for managing transactions in Microservices, its time to look at SAGA Pattern, another popular ways to manage distributed transactions. 

public class OrderSaga {

    @Autowired
    private OrderService orderService;

    @Autowired
    private PaymentService paymentService;

    @Autowired
    private InventoryService inventoryService;

    @SagaStart
    public


    public void placeOrder(Order order) throws Exception {
        try {
            // Step 1: Reserve inventory
            inventoryService.reserveInventory(order);
            order.setStatus("Inventory Reserved");

            // Step 2: Charge payment
            paymentService.chargePayment(order);
            order.setStatus("Payment Charged");

            // Step 3: Confirm order
            orderService.confirmOrder(order);
            order.setStatus("Order Confirmed");
        } catch (Exception ex) {
            // Step 4: Handle failure
            inventoryService.cancelInventoryReservation(order);
            paymentService.refundPayment(order);
            orderService.cancelOrder(order);
            order.setStatus("Transaction Failed");
        }
    }
}

In addition to the above approaches, there are a few more techniques and best practices that can help manage distributed transactions in microservices. 

Let's take a look at a few of them.

4. Use Idempotent Operations

Idempotent operations are operations that can be repeated without changing the outcome. In a distributed system, using idempotent operations can help ensure that the same operation is not performed twice, even if it is retried due to a failure. 

For example, if a service is trying to update a record in a database, it can check if the record already exists before performing the update operation. If the record already exists, the service can skip the update operation and return a success response.

5. Implement Retry and Timeout Mechanisms

In a distributed system, network failures and timeouts are common. Implementing retry and timeout mechanisms can help handle these failures gracefully. For example, if a service fails to respond to a request, the client can retry the request a few times before giving up. 

Similarly, if a service takes too long to respond, the client can timeout the request and handle the failure gracefully.

6. Use a Distributed Transaction Coordinator

A distributed transaction coordinator is a service that manages distributed transactions in a microservices architecture. It provides a centralized mechanism for coordinating the work of all the services involved in a transaction.

The coordinator can ensure that all the services commit or abort the transaction in a coordinated manner, even in the event of failures.

Let's take a look at a code example that implements idempotent operations and retry mechanisms to handle failures in a distributed transaction.

public class PaymentService {

    @Autowired
    private RestTemplate restTemplate;

    @Value("${inventory.service.url}")
    private String inventoryServiceUrl;

    @Value("${retry.maxAttempts}")
    private int maxAttempts;

    @Value("${retry.backoff}")
    private int backoff;

    @Retryable(value = { HttpClientErrorException.class }, 
maxAttempts = "${retry.maxAttempts}",
backoff = @Backoff(delay = "${retry.backoff}"))
    public void chargePayment(Order order) throws Exception {
        try {
            // Check if payment already exists
            Payment payment = restTemplate.getForObject(
"http://payment-service/payments/{orderId}", 
Payment.class, order.getId());
            if (payment != null) {
                // Payment already exists, skip the operation
                return;
            }

            // Perform payment
            restTemplate.postForObject("http://payment-service/payments",
        order, Payment.class);

        } catch (HttpClientErrorException ex) {
            // Handle 4xx errors
            if (ex.getStatusCode() == HttpStatus.CONFLICT) {
                // Payment already exists, skip the operation
                return;
            }
            throw ex;
        } catch (HttpServerErrorException ex) {
            // Handle 5xx errors
            throw ex;
        }
    }

    @Recover
    public void recoverChargePayment(HttpClientErrorException ex,
   Order order) {
        // Handle retry failure
        // Log the error and throw an exception
        throw new RuntimeException("Failed to charge payment after " 
        + maxAttempts + " retries");
    }
}

Conclusion

That's all about how to manage transactions in Microservices and distributed systems. Managing distributed transactions in a microservices architecture is a complex task that requires careful planning and implementation. 

In this article, we explored some of the popular approaches and best practices to manage distributed transactions, including idempotent operations, retry and timeout mechanisms, and the use of a distributed transaction coordinator.

You have also see the code example that implements idempotent operations and retry mechanisms to handle failures in a distributed transaction. 

By following these approaches and best practices, you can effectively manage distributed transactions in your microservices architecture and ensure that your system is reliable and scalable.

Other System Design and Microservices Tutorials and Resources

{ ACID, değişikliklerin bir veritabanına nasıl uygulanacağını yöneten 4 adet prensip sunar. Bunlar, Atomicity, Consistency, Isolation ve Durability prensipleridir. Bir kaç cümle ile açıklamak gerekirse;

Atomicity: En ksıa ifadesiyle ya hep, ya hiç. Arda arda çalışan transaction’lar için iki olası senaryo vardır. Ya tüm transaction’lar başarılı olmalı ya da bir tanesi bile başarısız olursa tümünün iptal edilmesi durumudur.

Consistency: Veritabanındaki datalarımızın tutarlı olması gerekir. Eğer bir transaction geçersiz bir veri üreterek sonuçlanmışsa, veritabanı veriyi en son güncel olan haline geri alır. Yani bir transaction, veritabanını ancak bir geçerli durumdan bir diğer geçerli duruma güncelleyebilir.

Isolation: Transaction’ların güvenli ve bağımsız bir şekilde işletilmesi prensibidir. Bu prensip sıralamayla ilgilenmez.Bir transaction, henüz tamamlanmamış bir başka transaction’ın verisini okuyamaz.

Durability: Commit edilerek tamamlanmış trasnaction’ların verisinin kararlı, dayanıklı ve sürekliliği garanti edilmiş bir ortamda (sabit disk gibi) saklanmasıdır. Donanım arızası gibi beklenmedik durumlarda transaction log ve alınan backup’lar da prensibe bağlılık adına önem arz etmektedir.

   [Suat KÖSE] yazısından alıntıdır}