win7搭建为伪分布式hadoop的步骤

这篇文章主要介绍“win7搭建为伪分布式hadoop的步骤”，在日常操作中，相信很多人在win7搭建为伪分布式hadoop的步骤问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”win7搭建为伪分布式hadoop的步骤”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！

创新互联公司是一家专注于成都网站设计、网站建设与策划设计,温县网站建设哪家好?创新互联公司做网站,专注于网站建设十载,网设计领域的专业建站公司;建站业务涵盖:温县等地区。温县做网站价格咨询:18982081108

window7下安装hadoop [32位]
1.下载hadoop
hadoop-2.2.0.tar.gz
解压
2.配置环境变量hadoop
3.修改%HADOOP_HOME%\etc\hadoop的hadoop-env.sh
export JAVA_HOME=D:\Java\jdk1.6.0_10#不能有空格
末尾添加[会出错就不添加]
set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop  
set YARN_CONF_DIR=%HADOOP_CONF_DIR%  
set PATH=%PATH%;%HADOOP_PREFIX%\bin 

下载hadoop-common-2.2.0-bin-master

把里面的文件拷贝到hadoop/bin下

4.配置core-site.xml



 
       
        fs.default.name  
        hdfs://0.0.0.0:19000  
       
   
 
5.配置hdfs-site.xml

 



     
       
        dfs.replication  
        1  
       
     

    
6.配置mapred-site.xml






     
      
        
         mapreduce.job.user.name  
         %USERNAME%  
        
      
        
         mapreduce.framework.name  
         yarn  
        
      
       
        yarn.apps.stagingDir  
        /user/%USERNAME%/staging  
       
      
       
        mapreduce.jobtracker.address  
        local  
       
      
     


7.创建yarn-site.xml文件



     
       
        yarn.server.resourcemanager.address  
        0.0.0.0:8020  
       
      
       
        yarn.server.resourcemanager.application.expiry.interval  
        60000  
       
      
       
        yarn.server.nodemanager.address  
        0.0.0.0:45454  
       
      
       
        yarn.nodemanager.aux-services  
        mapreduce_shuffle  
       
      
       
        yarn.nodemanager.aux-services.mapreduce.shuffle.class  
        org.apache.hadoop.mapred.ShuffleHandler  
       
      
       
        yarn.server.nodemanager.remote-app-log-dir  
        /app-logs  
       
      
       
        yarn.nodemanager.log-dirs  
        /dep/logs/userlogs  
       
      
       
        yarn.server.mapreduce-appmanager.attempt-listener.bindAddress  
        0.0.0.0  
       
      
       
        yarn.server.mapreduce-appmanager.client-service.bindAddress  
        0.0.0.0  
       
      
       
        yarn.log-aggregation-enable  
        true  
       
      
       
        yarn.log-aggregation.retain-seconds  
        -1  
       
      
       
        yarn.application.classpath  
        %HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*  
       
     

    切换到E:\hadoop\hadoop-2.2.0\etc\hadoop运行hadoop-env.cmd脚本，设置当前命令窗口执行环境变量
   执行 hdfs namenode -format
   16/02/27 11:36:23 INFO util.GSet: VM type       = 32-bit
16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB
16/02/27 11:36:23 INFO util.GSet: capacity      = 2^16 = 65536 entries
16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d
fs\name has been successfully formatted.
16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng
\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression
16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na
me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds
.
16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima
ges with txid >= 0
16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0
16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

到此，关于“win7搭建为伪分布式hadoop的步骤”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注创新互联网站，小编会继续努力为大家带来更多实用的文章！