Database Design and Data Consistency Strategies in Microservices

Problem Description: In a microservices architecture, each service typically owns an independent database, which introduces challenges of data distribution and consistency. Please elaborate on the principles of database design in microservices and detail how to address data consistency issues across services.

Knowledge Point Explanation:

1. Core Principle: Database Per Service

• Description: The core design principles of microservices architecture are loose coupling and high cohesion. This principle also applies to the data layer. Each microservice should own its private, independent database (or database schema) and can only operate on this data through its provided APIs. Other services cannot directly access this database.
• Benefits:
  • Loose Coupling: Services can independently choose the database technology (e.g., SQL, NoSQL) most suitable for their business needs, i.e., polyglot persistence.
  • Cohesion: Data is tightly bound to the business domain service it belongs to, making it easier to understand and maintain.
  • Independence: Services can independently change, scale, and deploy their database schema without affecting other services.

2. Challenge: Cross-Service Data Consistency

• Problem Origin: A business operation (e.g., placing an order) often requires updating data across multiple services (e.g., Order Service, Inventory Service, Account Service). In a monolithic application, this can be guaranteed through ACID transactions in the database. However, in microservices, each service's database is independent, making it impossible to use a single database transaction.
• Goal: We need a mechanism to guarantee eventual data consistency in the absence of distributed strong-consistency transactions (like Two-Phase Commit, 2PC, which is generally not adopted in microservices due to poor performance, high complexity, and low availability).

3. Solution: Saga Pattern

• Core Idea: The Saga pattern breaks down a distributed transaction spanning multiple services into a series of local transactions executed within individual services. Each local transaction commits and triggers the next one. If any step fails, the Saga executes a series of Compensating Transactions to undo the effects of the previously completed steps, thereby ensuring the system's eventual consistency.
• Two Coordination Patterns:

  • Choreography:

    1. Process: After executing its local transaction, each service publishes a Domain Event to a message broker (e.g., Kafka, RabbitMQ). Other related services subscribe to these events and trigger their own local operations.
    2. Example (Creating an Order):
       • Order Service: Creates an order (status "Pending"), publishes an OrderCreated event.
       • Inventory Service: Subscribes to the OrderCreated event, executes inventory deduction, publishes an InventoryUpdated event.
       • Account Service: Subscribes to the InventoryUpdated event, executes account deduction, publishes a PaymentCompleted event.
       • Order Service: Subscribes to the PaymentCompleted event, updates the order status to "Confirmed".
    3. Failure Handling: If the Account Service fails to deduct payment, it publishes a PaymentFailed event. The Inventory Service and Order Service subscribe to this event; the Inventory Service executes a compensating transaction (restores inventory), and the Order Service updates the order status to "Cancelled".
    4. Pros and Cons: Advantage is no central coordinator, leading to loose coupling; disadvantage is that the process logic is scattered across services, making it difficult to understand and debug.

  • Orchestration:

    1. Process: Introduce a centralized Saga Orchestrator. The orchestrator is responsible for receiving commands, sequentially calling each participant service, and handling responses and errors.
    2. Example (Creating an Order):
       • The Saga Orchestrator receives a "Create Order" command.
       • The orchestrator first calls the Order Service's "Create Pending Order" interface.
       • If successful, the orchestrator then calls the Inventory Service's "Deduct Inventory" interface.
       • If successful, the orchestrator calls the Account Service's "Deduct Payment" interface.
       • If all are successful, the orchestrator finally calls the Order Service's "Confirm Order" interface.
    3. Failure Handling: If the call to the Account Service fails, the orchestrator will execute compensating operations in the predefined order: first call the Inventory Service's "Restore Inventory" interface, then call the Order Service's "Cancel Order" interface.
    4. Pros and Cons: Advantage is that the process logic is centralized in the orchestrator, making it easy to manage and monitor; disadvantage is the introduction of a single point of responsibility and some coupling with the services.

4. Selection and Summary

• Choreography is suitable for simple processes with few participating services.
• Orchestration is suitable for complex processes with many participating services that require centralized control and have complex business logic. It is the more commonly used and recommended approach.
• Eventual Consistency: The Saga pattern guarantees eventual consistency. Before compensating transactions are completed, the system will be in a brief inconsistent state (e.g., the order is in "Pending" status, but inventory has already been deducted). This is a typical trade-off made in microservices architecture to achieve high availability and scalability.

By adopting patterns like Database Per Service and Saga, the microservices architecture can effectively manage distributed data consistency while maintaining service independence.