在 Linux 上启动 Hadoop,通常分为 单机模式、伪分布式模式、完全分布式模式。下面以**最常用、最典型的「伪分布式模式」**为例,说明启动步骤(Hadoop 3.x 为例,2.x 类似)。
Hadoop 依赖 Java:
java -version
如果没有 Java:
sudo apt install openjdk-8-jdk # Ubuntu
# 或
sudo yum install java-1.8.0-openjdk-devel
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh localhost # 测试是否能无密码登录
tar -zxvf hadoop-3.3.6.tar.gz
mv hadoop-3.3.6 /usr/local/hadoop
vi ~/.bashrc
添加:
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
使配置生效:
source ~/.bashrc
进入配置目录:
cd $HADOOP_HOME/etc/hadoop
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
hdfs namenode -format
start-dfs.sh
start-yarn.sh
或者一条命令全启动(不推荐新手):
start-all.sh
jps
应看到:
浏览器访问:
http://localhost:9870http://localhost:8088stop-dfs.sh
stop-yarn.sh
hadoop-env.sh 中设置:export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
如果你告诉我:
我可以给你更精确的启动方案。