Project objective

Customer requested to create a platform to provide Apache Kafka as a service for various development and integration teams. At the same time, they needed to map and integrate all operational processes associated with the creation of new instances, upgrades or connection to the central identity system. One of the requirements was running Apache Kafka in a Red Hat OpenShift environment.

The commercial variant of the Red Hat product AMQS was used, which extends Apache Kafka with interesting functionalities, such as upgrade automation or security configuration. At the same time, the Red Hat product is integrated into the OpenShift environment, including an operator that facilitates configuration into the environment.

One of the further requirements was to solve the UI for displaying data in Apache Kafka and the registry schema for the possibility of using AVRO schemas and maintaining the "life cycle" over these schemas.

Realization

The implementation took place by setting up the automation process of creating an Apache Kafka instance based on defined parameters. The process includes the following functionality:

• Creating an AMQS configuration in OpenShift using HELM
• Connecting the Cert Manager operator by configuring TLS server certificates
• Automated Kafka UI installation and configuration
• Automated installation and configuration registry scheme
• Existing systems connections – consultation and connections implementation
• Monitoring and alerting settings
• Creation of customer dashboards for the ability to monitor information
• Separation of operational and user parts (setting permissions, accesses)
• Security of graphical interfaces using IDM and central rights management

Project result

The result is a platform that provides customers with comprehensive functionality when using Apache Kafka, including the use of supporting components such as a graphical interface for Apache Kafka or a registry schema.

During the implementation and consultations, a connection was made to the original systems, when the compatibility of other schema registries (Confluent) was checked, and it was possible to successfully connect and use AVRO schemas during communication and at the same time enable users to display them using graphic tools.