Questa è una versione PDF del contenuto. Per la versione completa e aggiornata, visita:
https://blog.tuttosemplice.com/en/building-a-fintech-crm-event-driven-architecture-on-google-cloud/
Verrai reindirizzato automaticamente...
In today’s financial landscape, speed and reliability are not just features, but compliance requirements. Designing a Customer Relationship Management (CRM) system for the Fintech sector requires a paradigm shift from traditional monolithic systems based on relational databases and synchronous calls. In this technical guide, based on the development experience of the BOMA system, we will explore how an event-driven architecture on Google Cloud Platform (GCP) can solve the scalability, consistency, and responsiveness challenges typical of the industry.
A Fintech CRM is not limited to storing records. It must react in real-time to deposits, KYC (Know Your Customer) status changes, market fluctuations, and user interactions. A traditional Request/Response approach (synchronous HTTP) creates tight coupling between services, leading to bottlenecks and potential cascading failures.
The event-driven architecture (EDA) inverts this model. Instead of services calling each other directly, components emit “events” (facts that occurred, such as PaymentReceived or LeadCreated) that are consumed asynchronously by other services. According to Google Cloud Architecture documentation, this pattern drastically improves system resilience and scalability.
For the BOMA project, the choice of the technology stack fell on managed serverless services to minimize operational overhead and maximize scalability:
The beating heart of the architecture is Google Pub/Sub. Every significant action in the CRM is published as a message on a specific Topic.
Let’s imagine the flow of a new user registering:
user-onboarding topic.At this point, several Subscriptions trigger independent workers:
Best Technical Practice: In Fintech, event order is critical (you cannot withdraw funds before depositing them). We use Pub/Sub Ordering Keys (e.g., the user ID) to ensure that messages related to the same customer are processed in sequential order, while maintaining parallel scalability across different users.
The choice of Firestore over Cloud SQL is dictated by the need for real-time updates on the CRM operators’ dashboard. Firestore uses listeners (snapshot listeners) that allow the frontend to update automatically when a document changes, without the need for continuous polling.
Although Firestore is NoSQL, the data structure must be rigorous. A typical structure for a Fintech CRM might be:
/users/{userId}
- profileData (Map)
- kycStatus (String)
/transactions/{transactionId}
- amount (Number)
- currency (String)
- status (String)
- timestamp (Timestamp)Beware of Hotspotting: Avoid using timestamps or sequential IDs as document keys if you anticipate massive writes (>500/sec), as this concentrates the load on a single key range. Use randomly generated IDs or hashes.
The Cloud Functions act as the glue between Pub/Sub and Firestore. Each function is an atomic microservice with a single responsibility.
When a KYC check is completed, a KycCompleted event triggers a Cloud Function. This function:
PENDING to APPROVED.UserActive event to unlock trading features.This is the most critical section for a CTO or Lead Engineer. Distributed systems like Pub/Sub guarantee “at-least-once” delivery. This means that, rarely, your Cloud Function might receive the same payment event twice.
To avoid double charges or corrupted states, every operation must be idempotent. Here is how to implement it in Firestore:
eventId (generated at the source).eventId has already been processed in a support collection processed_events.eventId to the support collection, all atomically.This approach ensures the integrity of financial data even in the case of automatic retries by the Google infrastructure.
A CRM serves not only to manage but to understand. Operational data on Firestore is not optimized for complex analytical queries (e.g., “What is the average conversion rate by region in the last quarter?”).
For this, we implement a streaming pipeline to BigQuery. We can use the official “Stream Firestore to BigQuery” extension or a dedicated Cloud Function that listens for changes on Firestore and inserts data into partitioned tables on BigQuery.
This allows the Data Science team to analyze conversion funnels and user behavior without impacting the performance of the operational CRM database.
Building a Fintech CRM with an event-driven architecture on Google Cloud offers undeniable advantages in terms of decoupling and scalability. However, it shifts complexity from infrastructure management to application logic management (error handling, idempotency, eventual consistency).
By following the described patterns — rigorous use of Pub/Sub for buffering, Firestore for real-time state, and transactional idempotency checks — it is possible to create a robust system capable of handling the volumes and criticality of modern financial applications.
An event-driven architecture is fundamental in Fintech to ensure scalability and resilience, overcoming the limits of synchronous monolithic systems. This approach allows services to react in real-time to critical events like deposits or KYC status changes without creating tight dependencies between components. By using systems like Google Pub/Sub, traffic spikes are better managed, and the failure of a single service is prevented from blocking the entire platform.
Although Pub/Sub is designed for parallel scalability, chronological order is vital in the financial sector, for example, to process a deposit before a withdrawal. To solve this problem, Ordering Keys, such as the user ID, are used. This feature ensures that all messages related to the same customer are delivered and processed by workers in strict sequence, while maintaining parallel processing for different users.
Firestore is preferred over Cloud SQL in scenarios requiring real-time updates on operator dashboards. Thanks to snapshot listeners, the frontend updates automatically as data changes without having to perform continuous polling, reducing load and latency. However, attention must be paid to data modeling, avoiding sequential keys to prevent hotspotting problems during massive writes.
Idempotency is the property that guarantees that an operation produces the same result even if executed multiple times, which is essential to avoid double charges in case of message redelivery. In a GCP environment, it is implemented by verifying the existence of a unique event ID in a support collection within a Firestore transaction. If the ID is already present, the system ignores the event, protecting the integrity of financial data.
To perform complex analyses without impacting the performance of the operational Firestore database, a streaming pipeline to BigQuery is implemented. Using dedicated extensions or Cloud Functions, data is replicated in real-time in the data warehouse. This allows Data Science teams to analyze trends and conversion funnels on large amounts of historical data, keeping the CRM fast and responsive for end users.