下面给你一份**在 Ubuntu 上安装 Hadoop(单机/伪分布式)**的常见做法,适合学习和开发使用。
sudo apt update
sudo apt upgrade -y
sudo apt install openjdk-8-jdk -y
推荐 JDK 8(Hadoop 3.x 也支持 JDK 8/11)
检查:
java -version
配置 JAVA_HOME:
sudo nano /etc/environment
添加:
JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"
生效:
source /etc/environment
sudo adduser hadoop
sudo usermod -aG sudo hadoop
su - hadoop
以 Hadoop 3.3.6 为例:
wget https://downloads.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz
tar -xzvf hadoop-3.3.6.tar.gz
sudo mv hadoop-3.3.6 /usr/local/hadoop
nano ~/.bashrc
添加:
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
生效:
source ~/.bashrc
hadoop-env.shnano $HADOOP_HOME/etc/hadoop/hadoop-env.sh
确保:
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
core-site.xmlnano $HADOOP_HOME/etc/hadoop/core-site.xml
内容:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
hdfs-site.xmlnano $HADOOP_HOME/etc/hadoop/hdfs-site.xml
内容:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xmlcp $HADOOP_HOME/etc/hadoop/mapred-site.xml.template $HADOOP_HOME/etc/hadoop/mapred-site.xml
nano $HADOOP_HOME/etc/hadoop/mapred-site.xml
内容:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xmlnano $HADOOP_HOME/etc/hadoop/yarn-site.xml
内容:
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
sudo apt install ssh
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh localhost
hdfs namenode -format
start-dfs.sh
start-yarn.sh
jps
应看到:
浏览器访问:
http://localhost:9870http://localhost:8088stop-yarn.sh
stop-dfs.sh
如果你需要:
告诉我你的 Ubuntu 版本和使用场景,我可以进一步帮你定制。