In a Microservices architecture, services are designed to be small, autonomous, and loosely coupled. This design philosophy provides many benefits, including scalability, flexibility, and faster development cycles.
However, managing transactions across multiple microservices can be a challenging task. In this article, we will explore how to manage distributed transactions in microservices and provide relevant code examples.
There
are multiple ways to manage transactions in Microservices
like applying SAGA Pattern, using 2 Phase commit or using
Event Sourcing Design Pattern. In this article, we will understand
all of those but before that, let's understand what is distributed
transactions and what are challenges related to transaction
management in Microservices.
1. What are Distributed Transactions in Microservices?
A distributed transaction involves multiple services that need to work together to complete a transaction.
In
a Microservices architectureMicroservices
architecture,
each service may have its own database, and transactions that require
data from multiple services may need to coordinate the work of all
these services to ensure that the transaction is completed
successfully.
For example, when a user makes a purchase,
the transaction may involve several services, such as the order
service, payment service, and inventory service.
Challenges of Managing Distributed Transactions in Microservices
Managing
distributed transactions in a microservices architecture can be
challenging due to the following reasons:
1.
Complexity
Managing transactions across multiple services
can be complex, and coordinating the work of all these services can
be challenging.
2. Failure Handling
In a
distributed system, failures are inevitable, and handling these
failures in a distributed transaction can be challenging.
3.
Scalability
Microservices architecture is designed to be
scalable, and managing transactions across multiple services can be
challenging in a highly scalable environment.
4.
Latency
Coordinating the work of multiple services can
introduce latency, which can impact the performance of the system.
6 Ways to Manage Distributed Transactions in Microservices?
There are several approaches to manage distributed transactions in a microservices architecture. Here are some of the most popular approaches:
1. Two-Phase Commit (2PC)
Two-phase commit is a protocol used to ensure that all participants in a distributed transaction agree to commit or abort the transaction. In this approach, a coordinator service is responsible for coordinating the work of all the services involved in the transaction.
The coordinator asks all the services to prepare for the transaction, and if all the services are ready, the coordinator sends a commit message to all the services. If any of the services fail to prepare or respond, the coordinator sends an abort message to all the services.
Here is a nice diagram which shows how two phase commit works:
2. Saga Pattern
The Saga pattern is a pattern used to manage long-running transactions in a distributed system. In this pattern, each service involved in the transaction performs a local transaction and sends a message to the next service to perform its transaction.
If any of the services fail, the Saga can be rolled back by sending a compensating transaction to undo the work that has already been done.
Here is a nice diagram which shows how Saga pattern works in a Microservices architecture:
3. Event-Driven Architecture
Event-driven architecture is an architectural pattern that involves the use of events to trigger actions in a system. In a distributed transaction, each service can publish events when it has completed its part of the transaction.
Other services can then subscribe to these events and perform their part of the transaction. This is probably the simplest solution of managing distributed transactions in Microservices architecture.
Here is a nice diagram to show how Event Drive Architecture works:
Code Example of SAGA and 2-Phase Commit in Java Microservices
Let's take a look at some code examples of how to manage distributed transactions in microservices using the two-phase commit and Saga pattern approaches.
- Two-Phase Commit (2PC)
Here is a sample code of two phase commit using Java:
@Service public class OrderService { @Autowired private PaymentService paymentService; @Autowired private InventoryService inventoryService; @Transactional public void placeOrder(Order order) throws Exception { // Reserve inventory inventoryService.reserveInventory(order); // Charge payment paymentService.chargePayment(order); // Commit transaction // This is handled by the transaction manager } }
Here is how 2-phase commit look like in a sequence diagram:
- Saga Pattern
Now that we have seen code example of using 2 phase commit for managing transactions in Microservices, its time to look at SAGA Pattern, another popular ways to manage distributed transactions.
public class OrderSaga { @Autowired private OrderService orderService; @Autowired private PaymentService paymentService; @Autowired private InventoryService inventoryService; @SagaStart public public void placeOrder(Order order) throws Exception { try { // Step 1: Reserve inventory inventoryService.reserveInventory(order); order.setStatus("Inventory Reserved"); // Step 2: Charge payment paymentService.chargePayment(order); order.setStatus("Payment Charged"); // Step 3: Confirm order orderService.confirmOrder(order); order.setStatus("Order Confirmed"); } catch (Exception ex) { // Step 4: Handle failure inventoryService.cancelInventoryReservation(order); paymentService.refundPayment(order); orderService.cancelOrder(order); order.setStatus("Transaction Failed"); } } }
In addition to the above approaches, there are a few more techniques and best practices that can help manage distributed transactions in microservices.
Let's take a look at a few of them.
4. Use Idempotent Operations
Idempotent operations are operations that can be repeated without changing the outcome. In a distributed system, using idempotent operations can help ensure that the same operation is not performed twice, even if it is retried due to a failure.
For example, if a service is trying to update a record in a database, it can check if the record already exists before performing the update operation. If the record already exists, the service can skip the update operation and return a success response.
5. Implement Retry and Timeout Mechanisms
In a distributed system, network failures and timeouts are common. Implementing retry and timeout mechanisms can help handle these failures gracefully. For example, if a service fails to respond to a request, the client can retry the request a few times before giving up.
Similarly, if a service takes too long to respond, the client can timeout the request and handle the failure gracefully.
6. Use a Distributed Transaction Coordinator
A distributed transaction coordinator is a service that manages distributed transactions in a microservices architecture. It provides a centralized mechanism for coordinating the work of all the services involved in a transaction.
The
coordinator can ensure that all the services commit or abort the
transaction in a coordinated manner, even in the event of
failures.
Let's take a look at a code example that
implements idempotent operations and retry mechanisms to handle
failures in a distributed transaction.
public class PaymentService { @Autowired private RestTemplate restTemplate; @Value("${inventory.service.url}") private String inventoryServiceUrl; @Value("${retry.maxAttempts}") private int maxAttempts; @Value("${retry.backoff}") private int backoff; @Retryable(value = { HttpClientErrorException.class }, maxAttempts = "${retry.maxAttempts}", backoff = @Backoff(delay = "${retry.backoff}")) public void chargePayment(Order order) throws Exception { try { // Check if payment already exists Payment payment = restTemplate.getForObject( "http://payment-service/payments/{orderId}", Payment.class, order.getId()); if (payment != null) { // Payment already exists, skip the operation return; } // Perform payment restTemplate.postForObject("http://payment-service/payments", order, Payment.class); } catch (HttpClientErrorException ex) { // Handle 4xx errors if (ex.getStatusCode() == HttpStatus.CONFLICT) { // Payment already exists, skip the operation return; } throw ex; } catch (HttpServerErrorException ex) { // Handle 5xx errors throw ex; } } @Recover public void recoverChargePayment(HttpClientErrorException ex, Order order) { // Handle retry failure // Log the error and throw an exception throw new RuntimeException("Failed to charge payment after " + maxAttempts + " retries"); } }
Conclusion
That's all about how to manage transactions in Microservices and distributed systems. Managing distributed transactions in a microservices architecture is a complex task that requires careful planning and implementation.
In
this article, we explored some of the popular approaches and best
practices to manage distributed transactions, including idempotent
operations, retry and timeout mechanisms, and the use of a
distributed transaction coordinator.
You have also see the
code example that implements idempotent operations and retry
mechanisms to handle failures in a distributed transaction.
By following these approaches and best practices, you can effectively manage distributed transactions in your microservices architecture and ensure that your system is reliable and scalable.
Other System Design and Microservices Tutorials and Resources
{ ACID, değişikliklerin bir veritabanına nasıl uygulanacağını yöneten 4 adet prensip sunar. Bunlar, Atomicity, Consistency, Isolation ve Durability prensipleridir. Bir kaç cümle ile açıklamak gerekirse;
Atomicity: En ksıa ifadesiyle ya hep, ya hiç. Arda arda çalışan transaction’lar için iki olası senaryo vardır. Ya tüm transaction’lar başarılı olmalı ya da bir tanesi bile başarısız olursa tümünün iptal edilmesi durumudur.
Consistency: Veritabanındaki datalarımızın tutarlı olması gerekir. Eğer bir transaction geçersiz bir veri üreterek sonuçlanmışsa, veritabanı veriyi en son güncel olan haline geri alır. Yani bir transaction, veritabanını ancak bir geçerli durumdan bir diğer geçerli duruma güncelleyebilir.
Isolation: Transaction’ların güvenli ve bağımsız bir şekilde işletilmesi prensibidir. Bu prensip sıralamayla ilgilenmez.Bir transaction, henüz tamamlanmamış bir başka transaction’ın verisini okuyamaz.
Durability: Commit edilerek tamamlanmış trasnaction’ların verisinin kararlı, dayanıklı ve sürekliliği garanti edilmiş bir ortamda (sabit disk gibi) saklanmasıdır. Donanım arızası gibi beklenmedik durumlarda transaction log ve alınan backup’lar da prensibe bağlılık adına önem arz etmektedir.
[Suat KÖSE] yazısından alıntıdır}
