Reporting for PaaS: Transaction data streaming

Transaction (Business process) data streaming

Kafka Topic Description: transaction-topic

The Kafka topic transaction-topic serve as a central stream of events reflecting updates to transactions. Each message in the topic represents a new or updated TransactionRecord. This event-driven architecture ensures that every change in transaction data can be consumed in real-time by downstream services, enabling flexibility, scalability, and decoupled system design.

Event-driven Updates:

• Every update to a transaction (creation, status change, amount update, etc.) is published as an event in the topic.

• These events are immutable, meaning they represent the state of the transaction at the time of the update.

Message Format

Each message in the topic is a serialized representation of the TransactionRecord defined in the .proto schema. The payload includes all fields from the record, capturing the latest transaction state.

syntax = "proto3";

package sdk.finance.reports.messaging.proto;

message TransactionRecord {

  string id = 1;
  string transactionNumber = 2;
  string createdAt = 3; // Store as ISO-8601 string, e.g., "2024-10-28T14:30:00+02:00"
  string sourceCoinSerial = 4;
  string transactionType = 5;
  string transactionStatus = 6;
  string businessRequestStatus = 7;
  string senderName = 8;
  string senderId = 9;
  string receiverName = 10;
  string receiverId = 11;
  double amountSent = 12; // Representing BigDecimal
  double amountReceived = 13;
  double senderCommission = 14;
  double receiverCommission = 15;
  double totalCommission = 16;
  string sourceCurrency = 17;
  string destinationCurrency = 18;
  string destinationCoinSerial = 19;
}

Steps to Generate Classes

1. Place .proto file: Save the .proto file in the suitable directory (or the path you configured in protoSourceRoot).

2. Add Maven Plugin: Ensure the protobuf-maven-plugin is added to the pom.xml as shown in the provided configuration.

3. Run Maven Build: Execute the following command to trigger the code generation:

1. Generated Output: The generated Java classes will typically be in the target/generated-sources/protobuf/java directory, respecting the package.

Configuration:

<build>
        <extensions>
            <extension>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.6.0</version>
            </extension>
        </extensions>
        <plugins>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:4.28.3:exe:${os.detected.classifier}</protocArtifact>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.67.1:exe:${os.detected.classifier}</pluginArtifact>
                    <pluginId>grpc-java</pluginId>
                    <protoSourceRoot>${project.basedir}/src/main/resources/sdk/finance/proto</protoSourceRoot>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Multi-Language Support

While the example above is in Java, Protocol Buffers supports many other languages, such as Python, C#, Go, and more. To generate code in other languages:

1. Install the corresponding protoc plugin for the language.

2. Adjust the build tool configuration (e.g., Gradle, Bazel, or a language-specific build tool).