prkz.de - Blog


Keep it simple

Recovering access to a "lost" Apache Kafka Topic entry in Apache Zookeeper

written on 28 September 2020
I recently had the case of having the log segments of a (single-replicated) Kafka Topic still on disk and readable, but I lost all Zookeeper Data so that the topic was not visible to any Kafka client. First, after bringing the Kafka broker up again, i had to manually change the `meta.properties` to include the newly generated `cluster.id`. Now, the topics are stored in zookeeper under the path `/brokers/topics` I managed to re-create the Node for the "lost" Topic by first creating a dummy topic and copying and adjusting the Zookeeper node contents for that topic. On `zkCli.sh`: ```c create /brokers/topics/MyTopic {"version":2,"partitions":{"0":[1],"1":[1],"2":[1],"3":[1]},"adding_replicas":{},"removing_replicas":{}} ``` You can adjust the `partitions` as necessary for the number of partitions in your original topic data; the `1` for each partition is the broker id of the assigned kafka broker. Obviously, in setups, where not every broker has all partitions replicated, you will have to the configuration as needed. Now, the topic should simply be accessible in Kafka again, with all data: ```sh $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic MyTopic MyTopic:0:1011 MyTopic:1:1023 MyTopic:2:1099 MyTopic:3:1002 ```