Release Notes for Debezium 1.4
All notable changes for Debezium releases are documented in this file. Release numbers followSemantic Versioning.
Release 1.4.2.Final(March 1st, 2021)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.1 and has been tested with version 2.6.1 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.2.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.2.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.2.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
Breaking changes
In earlier versions of Debezium, the MySQL connector incorrectly emitted snapshot events using thec(create) operation type instead of the correct typer(read). If you have consumers which rely on that earlier behavior, you can use theio.debezium.connector.mysql.transforms.ReadToInsertEventsingle message transform to emulate that earlier behavior (DBZ-2788). A connector option which accidentally was introduced in 1.4.0 for this same purpose,snapshot.events.as.inserts, got removed again, and the SMT should be used instead in this situation. This SMT is meant for migration purposes only and will be removed in a future Debezium version.
The (incubating) Debezium connector for Oracle emits transaction ids in lower-case now, differing from the previous behavior of returning them as upper-case (DBZ-3165).
The previously deprecated snapshot modeINITIAL_SCHEMA_ONLYof the Oracle connector has been removed. Please useSCHEMA_ONLYinstead (DBZ-3034).
New Features
Make field descriptions consistent for time values (milliseconds, ms, sec, seconds, etc)DBZ-2858
Detect and skip non-parent index-organized tablesDBZ-3036
Capture additional JMX metrics for LogMinerDBZ-3038
SqlServerConnector does not implement validateDBZ-3056
Improve DML parser performanceDBZ-3078
Add ability to skip tests based on available database optionsDBZ-3110
Capture LogMiner session parameters when session fails to startDBZ-3153
Fixes
Wrong reference to KafkaConnector in setting up DebeziumDBZ-2745
Oracle Connector(Using Logminer) with Oracle RDS (v12) does not capture changesDBZ-2754
Oracle connector causes ORA-65090 when connecting to an Oracle instance running in non-CDB modeDBZ-2795
Warnings and notifications from PostgreSQL are ignored by the connector until the connection is closedDBZ-2865
ExtractNewRecord SMT incorrectly extracts ts_ms from source infoDBZ-2984
Replication terminates with ORA-01291: missing log fileDBZ-3001
Support multiple schemas with Oracle LogMinerDBZ-3009
Documentation
LOCK_TABLESshould beLOCK TABLESDBZ-3013Complete support for properties that contain hyphensDBZ-3019
SQLException for Global temp tables from OracleDatabaseMetaData.getIndexInfo() makes Debezium snapshotting failDBZ-3057
no viable alternative at input 'create or replace index'DBZ-3067
Strange transaction metadata for Oracle logminer connectorDBZ-3090
Environment Variables with spaces are truncated when written to properties fileDBZ-3103
Error: Supplemental logging not configured for table. Use command: ALTER TABLEDBZ-3109
Final stage of snapshot analyzes tables not present in table.include.list thus stumbles upon unsupported XMLTYPE tableDBZ-3151
Forever stuck with new binlog parser (1.3 and later) when processing big JSON column dataDBZ-3168
XStream does not process NUMER(1) dataDBZ-3172
Replace MySQL connector option with SMT for mitigating wrong op flagDBZ-2788
DML parser IndexOutOfRangeException with where-clause using "IS NULL"DBZ-3193
ORA-01284 file cannot be opened error when file locked by another processDBZ-3194
CommitThroughput metrics can raise division by zero errorDBZ-3200
Other changes
OSD certificationDBZ-2813
Integration with Service Registry promoted to GADBZ-2815
Use new deployment endpoint for releases to Maven CentralDBZ-3069
Remove zero-width whitespace from option namesDBZ-3087
Remove duplicate anchor links in Connector propertiesDBZ-3111
Config validation for OracleDBZ-3119
Clarify required privileges for using pgoutputDBZ-3138
Update Oracle documentationDBZ-3156
Put IIDR license requirement into NOTE boxDBZ-3163
Remove COLUMN_BLACK_LIST option in Oracle connectorDBZ-3167
Minor editorial update to PostgreSQL connector documentationDBZ-3192
Incorrect link/anchor pair for truncate.handling.mode property in PG properties documentationDBZ-3195
Release 1.4.1.Final(January 28th, 2021)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.1 and has been tested with version 2.6.1 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.1.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.1.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.1.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
New Features
Clarify information in Debezium connector for SQL Server docDBZ-2675
Add support for binary.handling.mode to the SQL Server connectorDBZ-2912
Use collation to get charset when charset is not setDBZ-2922
Additional logging for number and type of sql operationsDBZ-2980
Retry on "The server failed to resume the transaction"DBZ-2959
Fixes
Debezium Connectors are failing while reading binlog: Unknown event type 100DBZ-2499
Some column default values are not extracted correctly while reading table structureDBZ-2698
Supplemental logging is required for entire database rather than per monitored tableDBZ-2711
Missing log file error when current SCN differs from snapshotted in Oracle connector and LogminerDBZ-2855
GitHub action for "Build Testing Workflow" is using old artifacts and not building missing dependenciesDBZ-2861
Deadlock in the XStream handler and offset commiter call concurrentlyDBZ-2891
Sanitise DECIMAL string from VStreamDBZ-2906
Vitess Connector download link missing on websiteDBZ-2907
DML statements longer than 4000 characters are incorrectly combined from V$LOGMNR_CONTENTSDBZ-2920
Default database charset is not recordedDBZ-2921
Instable test: PostgresConnectorIT#testCustomSnapshotterSnapshotCompleteLifecycleHook()DBZ-2938
Snapshot causes ORA-08181 exceptionDBZ-2949
Postgres connector config validation fails because current connector is occupying replication slotDBZ-2952
Labeled create procedure’s body is not parsedDBZ-2972
Debezium swallows DML exception in certain casesDBZ-2981
Other changes
Migrate website build to HugoDBZ-575
Test binary/varbinary datatypesDBZ-2174
实现Scn domain typeDBZ-2518
Fix docs for message.key.columns and skipped.operationsDBZ-2572
Upgrade to Apache Kafka Connect 2.6.1DBZ-2630
Centralize postgres image name for test container testsDBZ-2764
Add missing connector options for Postgres connectorDBZ-2807
Importing TestDatabase as QuarkusTestResource for IT testsDBZ-2868
Set up Pulsar via Testcontainers in PulsarITDBZ-2915
Remove blacklist and whitelist from anchor link text in documentationDBZ-2918
不稳定的测试:PostgresShutdownIT # shouldStopOnPostgresFastShutdown()DBZ-2923
Rename whitelist/blacklist configs in examples to include/excludeDBZ-2925
Misspelling in readme for db2 connectorDBZ-2940
Fetch correct Apicurio version for ApicurioRegistryTestDBZ-2945
Incorrect link IDs in SQL Server connector snapshot metrics tableDBZ-2958
Release 1.4.0.Final(January 7th, 2021)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.0.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
Fixes
AWS RDS has different role names which make connector validation failDBZ-2800
Archive Log mining does not work with LogminerDBZ-2825
MySQL parser error for comments starting with tabDBZ-2840
连接器失败当使用的美元' sign in column name.DBZ-2849
Connection adapter not passed to Surefire testsDBZ-2856
Unsupported MariaDB syntax for generated columnsdbz - 2882
SLF4J API should not be included in Oracle distirbutionDBZ-2890
Vitess distro contains unaligned depsDBZ-2892
Changing base packages does not always trigger full buildsDBZ-2896
LogMiner causes DataException when DATE field is specified as NOT NULLDBZ-2784
Other changes
Remove LegacyDdlParser and related codeDBZ-2167
Add MongoDB connector interfaceDBZ-2808
sanitize.field.namessupport for Vitess ConnectorDBZ-2851Explicitly declare to Quarkus that ORM XML mapping is required for the outbox extensionDBZ-2860
Upgrade MySQL JDBC driver to 8.0.21DBZ-2887
Upgrade Guava library to 30.0DBZ-2888
Avoid exception when payload id field not presentDBZ-2889
Release 1.4.0.CR1(December 16th, 2020)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.0.CR1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.CR1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.CR1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
Other changes
Move Cassandra connector to separate repositoryDBZ-2636
Invalid column name should fail connector with meaningful messageDBZ-2836
Fix typos in downstream ModuleID declarations in monitoring.adocDBZ-2838
Duplicate anchor ID in partials/ref-connector-monitoring-snapshot-metrics.adocDBZ-2839
Fix additional typo in ModuleID declaration in monitoring.adocDBZ-2843
Edit modularization annotations in logging.adocDBZ-2846
Update Groovy version to 3.0.7DBZ-2850
Release 1.4.0.Beta1(December 9th, 2020)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
New Features
Add support for distributed tracingDBZ-559
Outbox Quarkus extension: Support OpenTracingDBZ-1818
Upgrade MongoDB driver to 4.x to run in native mode in GraalVM (for Quarkus extension)DBZ-2138
Allow snapshot records be generated either as create or read for MySQL connectorDBZ-2775
Support in Db2 connector for lowercase table and schema namesDBZ-2796
option to kill process when engine run crashesDBZ-2785
Add support for using Vitess primary key as Kafka message keyDBZ-2578
Add support for Nullable columnsDBZ-2579
Tablespace name LOGMINER_TBS should not be hardcoded in the Java codeDBZ-2797
Fixes
DDL parser: Allow stored procedure variables in LIMIT clauseDBZ-2692
Wrong mysql command in openshift dpeloyment docsDBZ-2746
long running transaction will be abandoned and ignoredDBZ-2759
MS SQL Decimal with default value not matching the scale of the column definition cause exceptionDBZ-2767
Cassandra Connector doesn’t shut down completelyDBZ-2768
MySQL Parser fails for BINARY collation shortcutDBZ-2771
PostgresConnectorIT.shouldResumeStreamingFromSlotPositionForCustomSnapshot is failing for wal2json on CIDBZ-2772
Connector configuration property "database.out.server.name" is not relevant for Logminer implementation but cannot be omittedDBZ-2801
CHARACTER VARYING mysql identifier for varchar is not supported in debeziumDBZ-2821
try-with-resources should not be used when OkHttp Response object is returnedDBZ-2827
EmbeddedEngine does not shutdown when commitOffsets is interruptedDBZ-2830
Rename user command parsing failsDBZ-2743
Other changes
Fix splitter annotations that control how content is modularized downstreamDBZ-2824
VerifyRecord#isValid() compares JSON schema twice instead of AvroDBZ-735
Don’t rely on deprecated JSON serialization functionality of MongoDB driverDBZ-1322
Move website build to GitHub ActionsDBZ-1984
Move Db2 connector to separate repositoryDBZ-2001
Modularize doc for SQL Server componentDBZ-2335
Upgrade apicurio to 1.3.2.FinalDBZ-2561
Remove obsolete logging files from /partials directoryDBZ-2740
Remove obsolete monitoring files from /partials directoryDBZ-2741
Increase Oracle CI frequencyDBZ-2744
Make Debezium example work with Podman instead of DockerDBZ-2753
Disable log mining history by defaultDBZ-2763
Upgrade -setup-java action to the latest 1.4.3DBZ-2770
Trigger non-core connector tests when core or DDL parser module are changedDBZ-2773
Add support for unsigned integer typesDBZ-2776
Update JDK action workflow matrix with JDK 16.0.0-ea.24DBZ-2777
Auto resolve latest JDK EA release numberDBZ-2781
Update content in modularized SQL Server connector docDBZ-2782
释放1.4.0.Alpha2(November 16th, 2020)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.0.Alpha2 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Alpha2 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Alpha2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
New Features
Move testcontainers changes on DebeziumContainer from UI PoC backend to Debezium main repoDBZ-2602
Add ability to map new name for the fields and headersDBZ-2606
Add close call to the Snapshotter interfaceDBZ-2608
Overriding Character Set MappingDBZ-2673
Support PostgreSQL connector retry when database is restartedDBZ-2685
Cassandra connector documentation typosDBZ-2701
Fix typo in converters docDBZ-2717
Add tests for DBZ-2617: PG connector does not enter FAILED state on failing heartbeatsDBZ-2724
DBZ-2662 Control ChangeEventQueue by the size in bytesDBZ-2662
Fixes
Oracle throw "no snapshot found based on specified time" when running flashback queryDBZ-1446
Exception when PK definition precedes column definitionDBZ-2580
Patroni can’t stop PostgreSQL when Debezium is streamingDBZ-2617
ChangeRecord informations don’t connect with the TableSchemaDBZ-2679
MySQL connector fails on a zero dateDBZ-2682
Oracle LogMiner doesn’t support partition tablesDBZ-2683
DB2 doesn’t start reliably in OCPDBZ-2693
Dropped columns cause NPE in SqlServerConnectorDBZ-2716
Timestamp default value in 'yyyy-mm-dd' format fails MySQL connectorDBZ-2726
Connection timeout on write should retryDBZ-2727
No viable alternative at input error on "min" columnDBZ-2738
SQLServer CI error in SqlServerConnectorIT.whenCaptureInstanceExcludesColumnsAndColumnsRenamedExpectNoErrors:1473DBZ-2747
debezium-connector-db2: DB2 SQL Error: SQLCODE=-206 on DB2 for z/OSDBZ-2755
no viable alternative at input 'alter table
orderdrop CONSTRAINT'dbz - 2760Tests are failing on macosDBZ-2762
Other changes
Move CI to Github Actions for all repositoriesDBZ-1720
Privileges missing from setup in documentation - Oracle LogMiner connectorDBZ-2628
Add validation that replication slot doesn’t existDBZ-2637
Update OpenJDK Quality Outreach jobsDBZ-2638
Re-unify monitoring content in the operations/monitoring.adoc fileDBZ-2659
Pull oracle specific changes for reading table column metadata into debezium-coreDBZ-2690
Intermittent test failure on CI - PostgresConnectorIT#shouldRegularlyFlushLsnWithTxMonitoringDBZ-2704
Topic routing doc formatting fixDBZ-2708
Re-unify logging content in the operations/logging.adoc fileDBZ-2721
Incorporate Oracle LogMiner implementation updatesDBZ-2729
Upgrade Vitess docker image to Vitess 8.0.0DBZ-2749
Intermittent SQL Server test failure on CI - SqlServerConnectorITDBZ-2625
Change initial.sync.max.threads to snapshot.max.threadsDBZ-2742
Release 1.4.0.Alpha1(October 22nd, 2020)
See thecomplete list of issues.
Kafka compatibility
This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See theKafka documentationfor compatibility with other versions of Kafka brokers.
Upgrading
Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.
当你决定升级其中一个连接器to 1.4.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.4.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.4.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.
If you are using our docker images then do not forget to pull them fresh from Docker registry.
New Features
Allow to specify subset of captured tables to be snapshottedDBZ-2456
Implement snapshot select override behavior for MongoDBDBZ-2496
Asciidoc block titles are rendered the same as regular textDBZ-2631
Allow closing of hung JDBC connectionDBZ-2632
Hide stacktrace when default value for SQL Server cannot be parsedDBZ-2642
Implement a CDC connector for VitessDBZ-2463
SqlServer - Skip processing of LSNs not associated with change table entries.DBZ-2582
Fixes
Cant override environment variablesDBZ-2559
Inconsistencies in PostgreSQL Connector DocsDBZ-2584
ConcurrentModificationException during exporting data for a mongodb collection in a sharded clusterDBZ-2597
Mysql connector didn’t pass the default db charset to the column definitionDBZ-2604
[Doc] "registry.redhat.io/amq7/amq-streams-kafka-25: unknown: Not Found" error occursDBZ-2609
[Doc] "Error: no context directory and no Containerfile specified" error occursDBZ-2610
SqlExceptions using dbz with Oracle on RDS online logs and logminerDBZ-2624
Mining session stopped - task killed/SQL operation cancelled - Oracle LogMinerDBZ-2629
Unparseable DDL: Using 'trigger' as table alias in view creationDBZ-2639
Antlr DDL parser fails to interpret BLOB([size])DBZ-2641
MySQL Connector keeps stale offset metadata after snapshot.new.tables is changedDBZ-2643
WAL logs are not flushed in Postgres ConnectorDBZ-2653
Debezium server Event Hubs plugin support in v1.3DBZ-2660
Cassandra Connector doesn’t use log4j for logging correctlyDBZ-2661
Should Allow NonAsciiCharacter in SQLDBZ-2670
MariaDB nextval function is not supported in grammarDBZ-2671
Sanitize field name do not santize sub struct fieldDBZ-2680
Debezium fails if a non-existing view with the same name as existing table is droppedDBZ-2688
Other changes
Merge MySQL doc source files into one againDBZ-2127
Metrics links duplicate anchor IDsDBZ-2497
Slim down Vitess container imageDBZ-2551
Modify release peipeline to support per-connector repos e.g. VitessDBZ-2611
Add Vitess connector to Kafka Connect container imageDBZ-2618
User Guide Documentation corrections for PostgreSQLDBZ-2621
Checkstyle should be built as a part of GH check formatting actionDBZ-2623
Upgrade MySQL JDBC driver to version 8.0.19DBZ-2626
Add support for multiple shard GTIDs in VGTIDDBZ-2635
Add documentation for Vitess connectorDBZ-2645
Restrict matrix job configurations to run only on SlavesDBZ-2648
Upgrade JUnit to 4.13.1DBZ-2658
Avoid parsing generated files in CheckstyleDBZ-2669
Update debezium/awestruct image to use Antora 2.3.4DBZ-2674
Fix doc typos and minor format glitches for downstream renderingDBZ-2681
Intermittent test failure on CI - RecordsStreamProducerIT#shouldReceiveHeartbeatAlsoWhenChangingNonWhitelistedTable()DBZ-2344
