你的问题应该是 “在 Debian 上如何配置 Kafka 消费者”(或“Debian 下 Kafka 消费端怎么配置”)。下面我分几种常见情况给你讲清楚。
sudo apt update
sudo apt install -y openjdk-17-jdk
java -version
wget https://downloads.apache.org/kafka/3.7.0/kafka_2.13-3.7.0.tgz
tar -xzf kafka_2.13-3.7.0.tgz
cd kafka_2.13-3.7.0
启动:
# 先启动 Zookeeper(老版本)
bin/zookeeper-server-start.sh config/zookeeper.properties
# 新版本(Kafka 3.x 可用 KRaft,不必须 Zookeeper)
bin/kafka-server-start.sh config/server.properties
bin/kafka-console-consumer.sh \
--bootstrap-server localhost:9092 \
--topic test \
--from-beginning
常用参数:
--group my-group:指定消费组--partition 0:指定分区--offset earliest:从最早开始consumer.properties 示例(Debian 上放哪都行):
bootstrap.servers=localhost:9092
group.id=my-consumer-group
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
auto.offset.reset=earliest
enable.auto.commit=true
auto.commit.interval.ms=5000
使用配置文件:
bin/kafka-console-consumer.sh \
--bootstrap-server localhost:9092 \
--topic test \
--consumer.config consumer.properties
pip install kafka-python
from kafka import KafkaConsumer
consumer = KafkaConsumer(
'test',
bootstrap_servers='localhost:9092',
group_id='my-group',
auto_offset_reset='earliest'
)
for msg in consumer:
print(msg.value.decode())
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>3.7.0</version>
</dependency>
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "my-group");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(List.of("test"));
while (true) {
var records = consumer.poll(Duration.ofMillis(100));
records.forEach(r -> System.out.println(r.value()));
}
监听地址
server.properties 里:
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://<服务器IP>:9092
防火墙
sudo ufw allow 9092
消费不到数据
auto.offset.reset=earliest我也可以给你写一个 Debian systemd 消费者服务,告诉我:
我可以直接帮你写完整配置。
如果你能补充一句:
是 命令行 / Python / Java / 容器 / systemd 哪种?
我可以给你更精准的 Debian 配置示例。