在 Debian 上启动 Hadoop,一般分为 单机/伪分布式 和 完全分布式 两种场景。下面以最常用、最容易验证的伪分布式(Pseudo-Distributed)模式为例,说明完整启动步骤。
sudo apt update
sudo apt install openjdk-11-jdk -y
java -version
sudo useradd -m hadoop
sudo passwd hadoop
su - hadoop
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh localhost
wget https://downloads.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz
tar -xzf hadoop-3.3.6.tar.gz
sudo mv hadoop-3.3.6 /opt/hadoop
配置环境变量(~/.bashrc):
export HADOOP_HOME=/opt/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
source ~/.bashrc
core-site.xml<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
hdfs-site.xml<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xml<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
hdfs namenode -format
start-dfs.sh
start-yarn.sh
jps
应看到:
浏览器访问:
http://localhost:9870http://localhost:8088测试命令:
hdfs dfs -mkdir /test
hdfs dfs -put ~/.bashrc /test
hdfs dfs -ls /test
stop-yarn.sh
stop-dfs.sh
如果你是要 集群模式(多节点) 或 Debian 12 + Hadoop 3.x 特殊问题,可以告诉我,我可以进一步说明。