An offline Kafka deployment needs more than just saved container images. The package has to match the target server, include the dependencies that server lacks, and expose host problems before startup. Now, that's a lot of things. Tons.

My involvement in Krate focuses on this practical layer (we can discuss unproven theories for days and weeks and nothing would eventually manifest, too many times I had tons of discussions and the next person just wouldn't understand). My overview of building Krate explains the broader project. Here, I want to look more closely at what makes a package usable after it leaves the development machine.

Match the package to the target processor

An early Krate fix addressed an ARM64 bundle being used on an x86 server. Yep, I know. What the hell. Before developing, outline all your target architectures that need support.

ARM64 and x86 are different processor architectures. A container image still contains programs compiled for a particular processor, packaging them in a container does not remove that requirement.

The fix needed to cover more than the initial download. The image export also had to select the intended platform. With containerd, one of Docker’s image-storage backends, simply pulling the desired architecture was not enough to make the packaging path reliable. Krate added explicit platform handling, an architecture marker in the bundle and a check against the installation host. Overkill? No. It's necessary.

For a user, the useful outcome is an early, understandable rejection. Discovering that a bundle targets the wrong processor before loading and starting its services is much easier to act on than interpreting several failing containers. Duh.

Make checksums work after the package moves

A checksum is a fingerprint of a file. Comparing it after a transfer helps detect whether the package changed or was damaged along the way.

Krate’s early checksum file included the absolute path used on the build machine. Once the archive moved to another server, verification could look for a file in a directory that did not exist there. Now this is where things get fancy. The package could be intact while the verification step still failed.

The correction was cheap, really cheap, if you put the core concepts of computer science into practice. Calculate the checksum from the output directory and record the archive’s filename without the builder’s private directory structure. That change made the accompanying checksum file portable. Done.

This is an easy detail to overlook in offline software distribution. The installation instructions need to work with the files the user receives, in their chosen directory, without depending on the builder’s filesystem.

Check the Linux host before changing it

A valid package can still meet an unsuitable host. Krate added preflight checks (do you think this only applies for aircrafts? No!) for Docker and Compose versions, occupied ports and firewall configuration. These checks help separate a host prerequisite from a Kafka problem. Maybe, in most cases, this surfaces the human problem. Anyways, moving on.

On Linux, several layers can affect container communication. One Krate change addressed Docker’s firewall zone, another detected conflicting arrangements between older and newer firewall tools. A reachable port alone does not establish that all the required container traffic can pass. If you think ping is enough, no.

RHEL packaging also needed attention to storage permissions and SELinux, the operating system’s additional access-control layer. A directory can exist and still be inaccessible to the container that needs to write into it. The package work included the ownership and mount-label handling needed for that environment.

I see these checks as part of the installation interface. They should name the relevant condition and the next step, rather than leaving the operator to assume every startup failure originates inside Kafka. Or worst, assuming every problem is a Kafka problem. They are also a reason to investigate the specific restriction instead of treating disabled host protections as the standard fix.

Include the dependencies of the dependencies (deep down the dependency tree we go!)

Bundling Docker packages does not automatically make a Docker installation offline-ready. Those packages can depend on operating-system components that are present on the build machine but missing from a minimal server.

That distinction matters in Krate’s RHEL work. The earlier package introduced installation handling, but its initial checks did not establish that the full cluster had booted on the target environment. A later change specifically tested offline dependency handling on a bare Rocky Linux 9.8 virtual machine, with network routing and DNS unavailable.

I had to include missing firewall-related packages and their supporting libraries. It also retained a useful control as the older bundle failed in that environment.

This is dependency closure in practical terms. Include what the selected packages require, including requirements introduced by another dependency, that may have another dependency. The offline test provides stronger evidence than a successful download on a connected machine. It still describes a tested environment, not every possible Linux installation.

Keep the supported Kafka editions clear

Krate’s history includes both ZooKeeper and KRaft setups. ZooKeeper is a separate coordination service while KRaft handles Kafka’s metadata coordination within Kafka itself. One must understand the difference.

The ZooKeeper package was frozen as a legacy path, while active development continued with KRaft and the tailored RHEL edition. Moving the KRaft setup to the Apache Kafka image also required changes to command locations and readiness checks. An image replacement affects the surrounding scripts, even when the service still has the same name.

The deployment boundary matters too. Multiple brokers on one virtual machine still share that machine. They do not provide protection against the whole host failing. When a host fails, time to call a human.

Offline Kafka deployment checklist

Before transferring and installing a package:

  • Confirm the target processor architecture and Linux distribution.
  • Verify the transferred archive using its accompanying checksum file.
  • Check Docker, Compose, disk space, required ports and host networking.
  • Check storage ownership and SELinux requirements where applicable.
  • Test required operating-system dependencies without relying on internet access.
  • Confirm which edition is actively maintained and what its deployment topology can tolerate.

Well, Krate does not end up to be a crater. LMFAO. I have met and spoke to enough humans for me to say, "Welp, had you put things into practice and get the right guidance....to be continued".