MogileFS的安装与配置
成都创新互联公司是一家业务范围包括IDC托管业务,网站空间、主机租用、主机托管,四川、重庆、广东电信服务器租用,四川移动机房托管,成都网通服务器托管,成都服务器租用,业务范围遍及中国大陆、港澳台以及欧美等多个国家及地区的互联网数据服务公司。
================================================================================
概述:
================================================================================
1.简介:
MogileFS是一个开源的分布式文件存储系统,由LiveJournal旗下的DangaInteractive公司开发。
Danga团队开发了包括Memcached、MogileFS、Perlbal等多个知名的开源项目。
目前使用MogileFS的公司非常多,如日本排名先前的几个互联公司及国内的yupoo(又拍)、digg、豆瓣、1号店、大众点评、搜狗和安居客等,分别为所在的组织或公司管理着海量的图片。
2.MogileFS特性:
★应用层
运行于用户空间,无需特殊的核心组件;
★无单点
MogStored:存储多个数据副本;
Tracker:多节点;
Database:MySQL(存放元数据);
★自动文件复制
★传输中立,无特殊协议,可通过NFS或HTTP进行通信
★简单的名称空间
diomain,class,fid(即访问文件的key);
每个文件对应一个key:用于domain定义名称空间;
★不用共享任何数据
元数据:DB
数据:MogStored,副本
★不需要RAID
3.MogileFS组件:
★MogileFS组件:
☉Tracker:追踪元数据
MogileFS的核心,是一个调度器;
服务进程为mogilefsd;
职责:删除、复制、监控、查询等;
☉MogStored:存储数据
mogstored进程: 数据存储的位置,通常是一个HTTP(WebDAV)服务器,用来数据的创建、删除、获取;
存储节点需定义“设备dev”用作存储空间;每个设备在当前集群中需通过一个唯一的DevID标识。
☉Database:存储元数据;
用于为tracker存储元数据信息,mogileFS
存储mogilefs的元数据,一般使用mysql,建议使用冗余方案保证可用性(MMM,MHA)
★MogileFS服务进程
mogilfsd (Tracker)
mogstored(MogStored)
4.MogileFS的实现机制和工作原理:
★实现机制
在该文件系统集群中,每一个数据块都会有多个副本,存放于多个Storage Node上,不同的文件系统副本的数量各不相同(一般建议为3份),在Tracker Node上会分别单独记录每一个数据块的存储位置(位于哪个Storage及路径)。若某个Storage节点不在线了,客户端向Tracker节点请求获取数据时,Tracker节点会返回用于冗余的副本的存储位置给客户端。这样就保证了整个文件系统中数据的高可用性。当有数据存储至某一storage节点后,各storage节点会根据Tracker的指示通过节点间通信将副本发送至其他节点上。若某一storage下线了,该节点上的副本会由其他节点重新提供,保证副本的个数不少于指定的值。
★工作原理
客户端需要存取数据时,首先与Tracker进行交互,Tracker通过查询数据库来获取客户端存取数据时所需要访问的元数据,然后返回给客户端,客户端利用Tracker返回的结果,与Storage Nodes节点进行交互完成数据存取(一次有可能联系一个也可能联系多个)。
附图:
5.MogileFS中的术语:
☉tracker
借助数据库保存各个节点文件的元数据信息, 便于检索定位数据位置并监控各个节点, 告知客户端存储区的位置并指挥storage节点复制数据副本, 进程为mogilefsd;
☉databases
为tracker节点存储节点文件的元数据信息;
☉storage
将指定域中的键转换为特有的文件名存储于在特定的设备文件中, 转换后文件名为值, storage自动维护键值之间的对应关系, storage节点使用http进行数据传输, 依赖于perbal, 进程为mogstored, perbal;
☉Domain
一个域中的键值是唯一的, 一个MogileFS可以有多个域来存储不同类型的文件,同一个Domain内Key必须唯一,不同的Domain内Key可以相同;
☉Class
一个域中的键值是唯一的, 一个MogileFS可以有多个域来存储不同类型的文件,同一个Domain内Key必须唯一,不同的Domain内Key可以相同;
☉device
一个存储节点, 可以有多个device, 就是用来存放文件的目录, 每个设备都有一个设备ID, 需要在mogstored配置文件中docroot配置, 设备不能删除, 只能将设备的状态置为dead, 置为dead之后数据就无法恢复了, 并且设备ID也无法再使用;
附图:
1.perl模块的安装方式:
★online(在线):
yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes
执行命令 : perl -MCPAN -e shell 进入 perl 依赖包安装模式 (或执行 cpan -i module_name::name)
cpan install App::cpanminus
cpan install MogileFS::Server
cpan install MogileFS::Utils
cpan install IO::AIO
cpan install IO::WrapTie
cpan install Danga::Socket★offline(线下):要大量解决依赖关系
$ perl Makefile.PL
$ make
$ sudo make install
★rpm
☉MogileFS安装:
tracker:
$ yum install MogileFS-Server-VERSION.rpm MogileFS-Server-mogilefsd-VERSION.rpm
mogstored:
$ yum install perl-IO-AIO
$ yum install MogileFS-Server-VERSION.rpm MogileFS-Server-mogstored-VERSION.rpm
实验:MofileFS的安装和配置
环境描述:
两台CentOS 7的虚拟主机node1和node2,node1即作为tracker又作为mogstore存储节点,同时还是mysql服务器;node2为tracker和mogstaor节点;
实验步骤:
1.首先在noge1上使用CPAN工具在线安装perl模块及mogilefs
[root@centos7 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes [root@centos7 ~]# perl -MCPAN -e shell cpan >install App::cpanminus cpan >install MogileFS::Server cpan >install MogileFS::Utils cpan >install IO::AIO cpan >install IO::WrapTie cpan >install Danga::Socket #========================================================================================= # 安装后各程序版本如下: [root@centos7 ~]# cpan Sys::Syscall Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT Sys::Syscall is up to date (0.25). [root@centos7 ~]# cpan MogileFS::Server Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT MogileFS::Server is up to date (2.72). [root@centos7 ~]# cpan MogileFS::Utils Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT MogileFS::Utils is up to date (2.29). [root@centos7 ~]# cpan IO::AIO Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT IO::AIO is up to date (4.34). [root@centos7 ~]# cpan IO::WrapTie Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT IO::WrapTie is up to date (2.111). [root@centos7 ~]# cpan Danga::Socket Reading '/root/.cpan/Metadata' Database was generated on Thu, 05 Jan 2017 18:41:02 GMT Danga::Socket is up to date (1.61).
2.在node1上安装MySQL(略),创建数据库,添加管理用户
[root@centos7 ~]# mysql -ptaoxiu Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 11 Server version: 5.5.44-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database mogfsdb; # 创建数据库 MariaDB [(none)]> grant all privileges on mogfsdb.* to 'moguser'@'127.0.0.1' identified by 'mogpass'; MariaDB [(none)]> grant all privileges on mogfsdb.* to 'moguser'@'192.168.1.113' identified by 'mogpass'; MariaDB [(none)]> flush privileges;
3.初始化数据库
[root@centos7 ~]# mogdbsetup --dbhost='127.0.0.1' --dbname=mogfsdb --dbuser=moguser --dbpass=mogpass --type=MySQL This will attempt to setup or upgrade your MogileFS database. It won't destroy existing data. Run with --help for more information. Run with --yes to shut up these prompts. Continue? [N/y]: y
查看数据库如下:
[root@centos7 ~]# mysql -ptaoxiu Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 5 Server version: 5.5.44-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | Syslog | | hellodb | | jingdong | | mogfsdb | | mysql | | ultrax | | zabbix | +--------------------+ 8 rows in set (0.12 sec) MariaDB [(none)]> use mogfsdb; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed MariaDB [mogfsdb]> show tables; # 生成的表 +----------------------+ | Tables_in_mogfsdb | +----------------------+ | checksum | | class | | device | | domain | | file | | file_on | | file_on_corrupt | | file_to_delete | | file_to_delete2 | | file_to_delete_later | | file_to_queue | | file_to_replicate | | fsck_log | | host | | server_settings | | tempfile | | unreachable_fids | +----------------------+ 17 rows in set (0.00 sec) MariaDB [mogfsdb]> exit
配置tracker节点,
1.在/etc目录下创建mogilefs目录,并提供配置文件mogilefsd.conf,如下:
[root@centos7 ~]# mkdir /etc/mogilefs [root@centos7 ~]# vim /etc/mogilefs/mogilefsd.conf # Enable daemon mode to work in background and use syslog daemonize = 1 # Where to store the pid of the daemon (must be the same in the init script) pidfile = /var/run/mogilefsd/mogilefsd.pid # 指定pid文件位置 # Database connection information db_dsn = DBI:mysql:mogfsdb:host=127.0.0.1 # 修改为上面授权的账号和密码 db_user = moguser # mysql用户 db_pass = mogpass # mysql密码 # IP:PORT to listen on for mogilefs client requests listen = 0.0.0.0:7001 # 监听的地址和端口(所有地址的7001端口) # Optional, if you don't define the port above. conf_port = 7001 # Number of query workers to start by default. query_jobs = 10 # 用于查询的进程数 # Number of delete workers to start by default. delete_jobs = 1 # 用于删除操作的进程数 # Number of replicate workers to start by default. replicate_jobs = 5 # 用于复制的进程数 # Number of reaper workers to start by default. # (you don't usually need to increase this) reaper_jobs = 1 # 用于回收资源的进程数 # Number of fsck workers to start by default. # (these can cause a lot of load when fsck'ing) #fsck_jobs = 1 # Minimum amount of space to reserve in megabytes # default: 100 # Consider setting this to be larger than the largest file you # would normally be uploading. #min_free_space = 200 # Number of seconds to wait for a storage node to respond. # default: 2 # Keep this low, so busy storage nodes are quickly ignored. #node_timeout = 2 # Number of seconds to wait to connect to a storage node. # default: 2 # Keep this low so overloaded nodes get skipped. #conn_timeout = 2 # Allow replication to use the secondary node get port, # if you have apache or similar configured for GET's #repl_use_get_port = 1
2.创建pid文件目录并赋予权限,然后启动服务。
# mogilefsd不能以root身份运行,必须用其他用户运行,为此添加mogilefs用户,然后启动服务 [root@centos7 ~]# useradd -r mogilefs # 创建pid文件目录并修改属主属组为mogilefs [root@centos7 ~]# mkdir /var/run/mogilefsd/ -p [root@centos7 ~]# chown -R mogilefs.mogilefs /var/run/mogilefsd/ # 启动服务 [root@centos7 mogilefs]# su - mogilefs -c "mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory # 查看端口7001 [root@centos7 mogilefs]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 25 *:514 *:* LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 128 *:7001 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 127.0.0.1:6010 *:* LISTEN 0 25 :::514 :::* LISTEN 0 128 :::22 :::* LISTEN 0 128 ::1:631 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 128 ::1:6010 :::*
接下来配置 mogstore节点,
1.编辑配置文件 vim /etc/mogilefs/mogstored.conf如下:
[root@centos7 ~]# vim /etc/mogilefs/mogstored.conf maxconns = 10000 # 最大并发连接数 httplisten = 0.0.0.0:7500 # 监听端口 mgmtlisten = 0.0.0.0:7501 # 管理端口,探测mogstore的健康状态 docroot = /data/mogstored # 数据存放位置
2.提供数据存放目录/data/mogstored,分区,格式化,挂载设备,并修改属主属组为mogilefs
[root@centos7 ~]# mkdir -pv /data/mogstored # 添加磁盘dev(生产环境中要把磁盘设备分区,创建文件系统,挂载到此目录下,这里为了测试创建目录dev) [root@centos7 ~]# mkdir /data/mogstored/dev{1,2} # 修改属主属组为mogilefs [root@centos7 ~]# chown -R mogilefs:mogilefs /data/mogstored/ [root@centos7 mogstored]# ll total 0 drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:52 dev1 drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:52 dev2 # 生产环境中分区,格式化,挂载文件系统,并授权 # fdisk /dev/sdb # mkfs.ext4 /dev/sdb1 # mkdir /mystore/dev1 -pv # mount /dev/sdb1 /mystore/dev1 # chown -R mogilefs.mogilefs /data/mogstored/dev1
3.启动服务,并查看端口7500,7501
[root@centos7 mogstored]# su - mogilefs -c "mogstored -c /etc/mogilefs/mogstored.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory [root@centos7 mogstored]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 25 *:514 *:* LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:7500 *:* LISTEN 0 128 *:7501 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 128 *:7001 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 127.0.0.1:6010 *:* LISTEN 0 25 :::514 :::* LISTEN 0 128 :::22 :::* LISTEN 0 128 ::1:631 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 128 ::1:6010 :::*
在节点node2上同样配置mogstore存储节点,如下:
1.同样适用CPAN在线安装perl相关模块和mogilefs,存储节点要解决依赖关系,安装perl-IO-AIO
[root@centos7 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes perl-IO-AIO [root@centos7 ~]# perl -MCPAN -e shell cpan >install App::cpanminus cpan >install MogileFS::Server cpan >install MogileFS::Utils cpan >install IO::AIO cpan >install IO::WrapTie cpan >install Danga::Socket
2.在/etc下创建目录,并提供mogstore的配置文件mogstored.conf
[root@centos7 ~]# mkdir /etc/mogilefs [root@centos7 ~]# vim /etc/mogilefs/mogstored.conf maxconns = 10000 httplisten = 0.0.0.0:7500 mgmtlisten = 0.0.0.0:7501 docroot = /data/mogstored
3.创建mogilefs用户,提供数据存放目录/data/mogstored,分区,格式化,挂载设备,并修改属主属组为mogilefs
[root@centos7 ~]# useradd -r mogilefs [root@centos7 ~]# mkdir -pv /data/mogstored/dev{3,4} # 两个节点设备不能相同,且为小写 mkdir: created directory ‘/data/mogstored’ mkdir: created directory ‘/data/mogstored/dev3’ mkdir: created directory ‘/data/mogstored/dev4’ [root@centos7 ~]# chown -R mogilefs:mogilefs /data/mogstored/ [root@centos7 ~]# cd /data/mogstored/ [root@centos7 mogstored]# ll total 0 drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:22 dev3 drwxr-xr-x 2 mogilefs mogilefs 6 Jan 5 15:22 dev4
4.启动服务,查看端口
[root@centos7 ~]# su - mogilefs -c "mogstored -c /etc/mogilefs/mogstored.conf --daemon" su: warning: cannot change directory to /home/mogilefs: No such file or directory [root@centos7 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 50 *:3306 *:* LISTEN 0 128 *:7500 *:* LISTEN 0 128 *:7501 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 127.0.0.1:631 *:* LISTEN 0 100  
标题名称:MogileFS的安装与配置
网页链接:http://cdkjz.cn/article/gioijs.html