Streaming Change Data Capture

Chapter 5. Architectural Planning and Implementation

To guide your approach to architectural planning and implementation, we have built the Replication Maturity Model, shown in Figure 5-1. This summarizes the replication and change data capture (CDC) technologies available to your IT team and their impact on data management processes. Organizations and the technologies they use generally fall into the following four maturity levels: Basic, Opportunistic, Systematic, and Transformational. Although each level has advantages, IT can deliver the greatest advantage to the business at the Transformational level. What follows is a framework, adapted from the Gartner Maturity Model for Data and Analytics (ITScore for Data and Analytics, October 23, 2017—see Appendix A), which we believe can help you steadily advance to higher levels.

Figure 5-1. Replication Maturity Model

Level 1: Basic

At the Basic maturity level, organizations have not yet implemented CDC. A significant portion of organizations are still in this phase. During a course on data integration at a TDWI event in Anaheim in Orlando in December 2017, this author was surprised to see only half of the attendees raise their hands when asked if they used CDC.

Instead, organizations use traditional, manual extract, transform, and load (ETL) tools and scripts, or open source Sqoop software in the case of Hadoop, that replicate ...