2 posts tagged with "CDC"

Change Data Capture (CDC) is a vital process for tracking and propagating changes in databases in real-time. Debezium, an open-source platform from Red Hat, stands as a powerful tool in this domain, capturing row-level changes and streaming them as events.

This article focuses on the integration of Debezium within SDLB and demonstrates how it can be leveraged for change data capture and processing.

In many cases datasets have no constant live. New data points are created, values changed and data expires. We are interested in keeping track of all these changes. This article first presents collecting data utilizing JDBC and deduplication on the fly. Then, a Change Data Capture (CDC) enabled (MS)SQL table will be transferred and historized in the data lake using the Airbyte MS SQL connector supporting CDC. Methods for reducing the computational and storage efforts are mentioned.