Reporting for PaaS: Transaction data streaming

28. 01. 2025

At SDK.finance, we record transaction data in Kafka, ensuring reliable storage and real-time access. This data can be utilized to build comprehensive reports, enabling your business to make informed decisions.

This solution is perfect for automating reporting, tracking financial operations, and analyzing data, making it an essential component for efficient transaction management in modern fintech applications.

Kafka Topic Description: `transaction-topic`

The Kafka topic transaction-topic serve as a central stream of events reflecting updates to transactions. Each message in the topic represents a new or updated TransactionRecord. This event-driven architecture ensures that every change in transaction data can be consumed in real-time by downstream services, enabling flexibility, scalability, and decoupled system design.

Event-driven Updates:

Every update to a transaction (creation, status change, amount update, etc.) is published as an event in the topic.
These events are immutable, meaning they represent the state of the transaction at the time of the update.

Message Format

Each message in the topic is a serialized representation of the TransactionRecord defined in the .proto schema. The payload includes all fields from the record, capturing the latest transaction state.

syntax = "proto3"; package sdk.finance.reports.messaging.proto; message TransactionRecord { string id = 1; string transactionNumber = 2; string createdAt = 3; // Store as ISO-8601 string, e.g., "2024-10-28T14:30:00+02:00" string sourceCoinSerial = 4; string transactionType = 5; string transactionStatus = 6; string businessRequestStatus = 7; string senderName = 8; string senderId = 9; string receiverName = 10; string receiverId = 11; double amountSent = 12; // Representing BigDecimal double amountReceived = 13; double senderCommission = 14; double receiverCommission = 15; double totalCommission = 16; string sourceCurrency = 17; string destinationCurrency = 18; string destinationCoinSerial = 19; }

Generating Java Classes from the `.proto` File

With the Maven plugin configuration provided (protobuf-maven-plugin), you can generate Java classes corresponding to the TransactionRecord schema. The generated classes provide getters and setters for all fields, as well as efficient serialization and deserialization methods.

Steps to Generate Classes

Place .proto file: Save the .proto file in the suitable directory (or the path you configured in protoSourceRoot).
Add Maven Plugin: Ensure the protobuf-maven-plugin is added to the pom.xml as shown in the provided configuration.
Run Maven Build: Execute the following command to trigger the code generation:

mvn clean compile
Generated Output: The generated Java classes will typically be in the target/generated-sources/protobuf/java directory, respecting the package.

Configuration:

    <build>

        <extensions>

            <extension>

                <groupId>kr.motd.maven</groupId>

                <artifactId>os-maven-plugin</artifactId>

                <version>1.6.0</version>

            </extension>

        </extensions>

        <plugins>

            <plugin>

                <groupId>org.xolstice.maven.plugins</groupId>

                <artifactId>protobuf-maven-plugin</artifactId>

                <version>0.6.1</version>

                <configuration>

                    <protocArtifact>com.google.protobuf:protoc:4.28.3:exe:${os.detected.classifier}</protocArtifact>

                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.67.1:exe:${os.detected.classifier}</pluginArtifact>

                    <pluginId>grpc-java</pluginId>

                    <protoSourceRoot>${project.basedir}/src/main/resources/sdk/finance/proto</protoSourceRoot>

                </configuration>

                <executions>

                    <execution>

                        <goals>

                            <goal>compile</goal>

                        </goals>

                    </execution>

                </executions>

            </plugin>

        </plugins>

    </build>

Multi-Language Support

While the example above is in Java, Protocol Buffers supports many other languages, such as Python, C#, Go, and more. To generate code in other languages:

Install the corresponding protoc plugin for the language.
Adjust the build tool configuration (e.g., Gradle, Bazel, or a language-specific build tool).