资讯

精准传达 • 有效沟通

从品牌网站建设到网络营销策划,从策略到执行的一站式服务

impala大数据量查询/tmp/impala-scratch创建异常

使用impala做大数据量查询distinct的时候报如下错误

网站设计制作过程拒绝使用模板建站;使用PHP+MYSQL原生开发可交付网站源代码;符合网站优化排名的后台管理系统;网站建设、做网站收费合理;免费进行网站备案等企业网站建设一条龙服务.我们是一家持续稳定运营了十载的成都创新互联公司网站建设公司。

WARNINGS: Create file /tmp/impala-scratch/24414dab2c19caca:e54b206c5ab149d_24414dab2c19caca:e54b206c5ab149f_91001337-9d70-4c93-84ce-e7916c1ae804 failed with errno=2 description=Error(2): No such file or directory
Backend 4:Create file /tmp/impala-scratch/24414dab2c19caca:e54b206c5ab149d_24414dab2c19caca:e54b206c5ab149f_91001337-9d70-4c93-84ce-e7916c1ae804 failed with errno=2 description=Error(2): No such file or directory

查阅文档知道impala在大数据量处理时会用到磁盘保存数据

  • By default, intermediate files used during large sort, join, aggregation, or analytic function operations are stored in the directory /tmp/impala-scratch. These files are removed when the operation finishes. (Multiple concurrent queries can perform operations that use the "spill to disk" technique, without any name conflicts for these temporary files.) You can specify a different location by starting the impalad daemon with the --scratch_dirs="path_to_directory" configuration option or the equivalent configuration option in the Cloudera Manager user interface. You can specify a single directory, or a comma-separated list of directories. The scratch directories must be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts, depending on the capacity and speed of the available storage devices. Impala will not start if it cannot create or read and write files in the "scratch" directory. If there is less than 1 GB free on the filesystem where that directory resides, Impala still runs, but writes a warning message to its log.

直接到各个impalad节点上创建/tmp/impala-scratch目录,并赋予读写权限

 mkdir /tmp/impala-scratch
 chmod 777 /tmp/impala-scratch

分享名称:impala大数据量查询/tmp/impala-scratch创建异常
网站网址:http://cdkjz.cn/article/jdhsde.html
多年建站经验

多一份参考,总有益处

联系快上网,免费获得专属《策划方案》及报价

咨询相关问题或预约面谈,可以通过以下方式与我们联系

大客户专线   成都:13518219792   座机:028-86922220