The Debezium team is excited to announce the first release candidate of Deebzium 2.2, Debezium2.2.0.CR1.
This release primarily focuses on stability improvements and bug fixes; however, there are a number of new features and breaking changes. In this release, Debezium migrated to Quarkus 3.0.0.Final, there are performance improvements to Debezium Server Pulsar sink, Jolokia can be enabled inside Debezium’s Kafka Connect container image, incubating support for incremental snapshots on MongoDB multi-replica and sharded clusters, and the deprecation usage of Docker Hub for images.
Let’s take a moment and dive into several of these and what it means moving forward!
Upgrade to Quarkus 3
Quarkus is a Kubernetes Native Java stack that combines the best Java libraries to create fast, low footprint applications. The Debezium Server runtime is based on Quarkus as well as part of Debezium UI. Additionally, the Debezium Outbox extension is also based on the Quarkus platform.
The upgrade to Quarkus 3 introduces a number of improvements, including using the latest stable releases of a plethora of Java libraries, including the migration from Java EE to Jakarta EE. If you are not familiar with this migration, previously most Java EE platform classes were bundled in the packagejavax.*
. Over the past year or two, more applications have started the move from JavaEE or J2EE to Jakarta EE, and Quarkus 3 marks this transition era. Overall, the only real change is that classes that previously resided injavax.*
now are placed injakarta.*
.
If your application makes use of the Debezium Quarkus Outbox extension, be aware that in order to use Debezium 2.2 with Quarkus, you will need to migrate to Quarkus 3. This also means that if you want to take advantage of the Outbox extension for Reactive data sources, you will be required to use Quarkus 3 as well.
Finally, if you are developing or maintaining sink adapters for Debezium Server, you will also need to make adjustments to using the new Jakarta EE annotations rather than the older Java EE annotations.
Debezium Server Pulsar Changes
In prior versions of the Debezium Server Pulsar sink, the adapter leveraged thesend()
method to deliver messages in a synchronous way. While this works for sending one-off messages, this has the potential to introduce connector latency as the method waits an acknowledgement of send operation sequentially. Since the Debezium Server sink adapters are provided a collection of events to deliver, the synchronous nature just does not perform well.
Starting Debezium 2.2, the Pulsar sink will now usesendAsync()
to asynchronously deliver the batch of events to Pulsar, netting a substantial increase in overall throughput. While each event within the batch is delivered asynchronously, the adapter will only proceed to the next batch once the current batch is acknowledged in entirety.
Jolokia support
Jolokia is a JMX-HTTP bridge that provides an alternative to using JSR-160 to gather metrics. It is an agent based approach that improves traditional JMX by introducing unique features like bulk requests and fine-grained security policies.
With Debezium 2.2, thedebezium/connect
image now ships with Jolokia, but this agent isn’t enabled by default. In order to enable Jolokia support, the container must be started withENABLE_JOLOKIA
set totrue
. By default, Jolokia will bind to port 8778 when enabled.
In the event that a different port is required, Jolokia will need to be enabled differently. For example, in order to enable Jolokia using port 9779, do not set theENABLE_JOLOKIA
but instead configure theKAFKA_OPTS
environment variable as follows:
-e KAFKA_OPTS="-javaagent:$(ls "$KAFKA_HOME"/libs/jolokia-jvm-*.jar)=port=9779,host=*"
By specifying the above environment variable, Jolokia’s JMX-HTTP bridge will be available on port 9779 of the container.
Do not forget to add the Jolokia port to the container’s list of exposed ports when starting. |
Deprecation of Docker Hub
Docker recently announced a reduction in their free organization account offerings, which is leveraged by a number of open-source communities, including Debezium. Unfortunately, Debezium does not qualify according to their rules.
Despite that Docker chose to walk back their decision, the Debezium team believes that we need a solid path forward to guarantee the availability of our images to our community without disruption. Debezium has been dual-publishing to bothdocker.io
andquay.io
for quite some time, and we will continue this for Debezium 2.2.0.Final and for all Debezium 2.3previewreleases.
We plan to publish a blog post going into more detail in the coming days with more details. In the meantime, all users should plan to migrate toquay.io
as soon as possible to avoid disruption of fetching newer versions of Debezium.
Other fixes
There were quite a number of other improvements, bug fixes, and stability changes in this release, some noteworthy are:
Upgrade dependencies (Quarkus, etc) of Debezium UIDBZ-4109
Failed retriable operations are retried infinitelyDBZ-4488
UI- Add the UI to configure the additional properties for a connectorDBZ-5365
Capture events in order across mongodb shardsDBZ-5590
DDL events not stored in schema history topic for excluded tablesDBZ-6070
Oracle path used current batchSize to calculate end scn is wrong, need to use min batch sizeDBZ-6155
Upgrade UI build to use Debezium 2.2 or latestDBZ-6173
Oracle-Connector dbz##user needs more rightsDBZ-6198
Make quay.io primary image repositoryDBZ-6216
Multiplatform build of example-postres failsDBZ-6258
Add protoc version property to postgres connector pom.xmlDBZ-6261
Pass through configurations for kafka topics/configurationDBZ-6262
Postgres connector doesn’t need logical WAL level when snapshotting onlyDBZ-6265
Update config properties in RHEL deployment instructionsDBZ-6266
MySQL connector doesn’t need to query binlog when snapshotting onlyDBZ-6271
Table names with spaces are not correctly deserialized when using an Infinispan cache as the transaction bufferDBZ-6273
Infinispan cache configuration used by Oracle tests are not compatible with Infinispan 14.0.2DBZ-6274
Transaction buffer state can become corrupted when using Infinispan cache with LOBsDBZ-6275
Enable the docker tag to be configurable in the Spanner connectorDBZ-6302
Upgrade MySQL JDBC driver to 8.0.32DBZ-6304
Allow specifying docker image reference in MongoDB testcontainers implementationDBZ-6305
UseMongoDbContainerinstead ofMongoDBContainertest containers class in ConnectorConfiguration classDBZ-6306
DDL statement couldn’t be parsed - Oracle connector 2.1.3.FinalDBZ-6314
Unparsable DDL statements (MySQL/MariaDB)DBZ-6316
Remove outdated information about SYS user accounts with OracleDBZ-6318
Cassandra 3 cannot be built using JDK20DBZ-6320
Bundle Jolokia with Debezium connect imageDBZ-6323
Altogether,34 issueswere fixed for this release. A big thank you to all the contributors from the community who worked on this release:Anisha Mohanty,Bob Roldan,Chris Cranford,Harvey Yue,Jacob Gminder,Jiri Pechanec,Jochen Schalanda,Mario Fiore Vitale,Mark Bereznitsky,Ondrej Babec,Pengwei Dou,Robert Roldan, andVojtech Juranek!
Outlook & What’s Next?
我们正在接近2.2 Debezium rel结束开云体育官方注册网址ease cycle, with a final release expected this week. If there are any outstanding bugs or issues, please get in touch with us so that can be addressed prior to the final release.
Debe开云体育官方注册网址z Debezium团队转移他们的注意力ium 2.3. The Debezium 2.3 release will be a much more condensed and focused release, as our goal is to release it in late June. The Debezium roadmap has been updated and the following features are planned for this quarter:
支持可配置的信号通道
Support exactly once delivery semantics (phase 1)
Kubernetes operator for Debezium Server
Oracle OpenLogReplicator adapter proof-of-concept / incubating implementation
Debezium UI improvements
We would like to hear your feedback or suggestions, so if you have anything you’d like to share be sure to get in touch with us on themailing listor ourchat.
Until next time, let the changes continue to stream…
About Debezium
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top ofKafkaand providesKafka Connectcompatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium isopen sourceunder theApache License, Version 2.0.
Get involved
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter@debezium,chat with us on Zulip, or join ourmailing listto talk with the community. All of the code is open sourceon GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know orlog an issue.