下面给你一个**在 Debian 上搭建 Kafka 集群(多 broker)**的简明实战步骤,假设你用 3 台 Debian 机器,Kafka 依赖 Zookeeper(先以传统方式为例)。
192.168.1.101 kafka1
192.168.1.102 kafka2
192.168.1.103 kafka3
Kafka 需要 JDK 8 或以上(推荐 JDK 11)
sudo apt update
sudo apt install -y openjdk-11-jdk
java -version
cd /opt
sudo wget https://downloads.apache.org/kafka/3.6.1/kafka_2.13-3.6.1.tgz
sudo tar -xzf kafka_2.13-3.6.1.tgz
sudo mv kafka_2.13-3.6.1 kafka
Kafka 3.x 仍可用 Zookeeper(KRaft 是新模式)
编辑:
/opt/kafka/config/zookeeper.properties
dataDir=/var/lib/zookeeper
clientPort=2181
maxClientCnxns=0
server.1=192.168.1.101:2888:3888
server.2=192.168.1.102:2888:3888
server.3=192.168.1.103:2888:3888
sudo mkdir -p /var/lib/zookeeper
echo 1 | sudo tee /var/lib/zookeeper/myid # kafka1
echo 2 | sudo tee /var/lib/zookeeper/myid # kafka2
echo 3 | sudo tee /var/lib/zookeeper/myid # kafka3
编辑:
/opt/kafka/config/server.properties
broker.id=1 # kafka1=1, kafka2=2, kafka3=3
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://192.168.1.101:9092
log.dirs=/var/lib/kafka-logs
num.partitions=3
default.replication.factor=3
min.insync.replicas=2
zookeeper.connect=192.168.1.101:2181,192.168.1.102:2181,192.168.1.103:2181
创建目录:
sudo mkdir -p /var/lib/kafka-logs
/opt/kafka/bin/zookeeper-server-start.sh -daemon /opt/kafka/config/zookeeper.properties
/opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties
/opt/kafka/bin/zookeeper-shell.sh 192.168.1.101:2181
ls /brokers/ids
/opt/kafka/bin/kafka-topics.sh \
--create --topic test \
--partitions 3 --replication-factor 3 \
--bootstrap-server 192.168.1.101:9092
/opt/kafka/bin/kafka-topics.sh --describe --topic test --bootstrap-server 192.168.1.101:9092
systemd 管理 Kafka / Zookeeper如果你愿意,我可以:
你现在是测试还是生产环境?