Difference between revisions of "CentOS 8.x Moosefs 3.x Initial setup"
m |
m |
||
Line 1: | Line 1: | ||
[[Main Page|Home]] > [[CentOS]] > [[CentOS 8.x]] > [[CentOS 8.x Distributed Filesystems]] > [[CentOS 8.x Moosefs 3.x]] > [[CentOS 8.x Moosefs 3.x Initial setup]] | |||
==Theory== | ==Theory== | ||
Line 221: | Line 220: | ||
[[Main Page|Home]] > [[CentOS]] > [[CentOS 8.x]] > [[CentOS 8.x Distributed Filesystems]] > [[CentOS 8.x Moosefs 3.x]] > [[CentOS 8.x Moosefs 3.x Initial setup]] |
Latest revision as of 17:33, 15 May 2022
Home > CentOS > CentOS 8.x > CentOS 8.x Distributed Filesystems > CentOS 8.x Moosefs 3.x > CentOS 8.x Moosefs 3.x Initial setup
Theory
To setup moosefs community edition you need following types of servers:
- master
- This is the master server which coordinates all activities within the distributed filesystem among various nodes. This maintains metadata about files (Which file / chunk is at which node).
- metalogger
- This server can keep track of changes to metadata. If master goes down we should be able to bring up another master using this server
- chunk servers
- These nodes store data and contribute to storage space / throughput. Chunk servers can specify a mounted filesystem or device that can be used by moosefs for storing data
- clients
- These nodes mount the distributed file-system. Multiple clients can mount the filesystem for rw concurrently without any issue.
Initialization
For moosefs to work above nodes (master, metalogger, chunkservers, etc.) should resolve each other by name. That can be done by DNS or by /etc/hosts file. For purpose of this document assume following names and IPs are present in /etc/hosts in all nodes:
172.31.1.183 mfsmaster 172.31.1.184 mfsmetalogger 172.31.1.185 mfschunk1 172.31.1.186 mfschunk2 172.31.1.187 mfschunk3 172.31.1.188 mfsclient1
Here name mfsmaster is critical as it is default in most configuration files. If you change this then please plan to change the name appropriately in most configuration files on most nodes.
Master setup
To setup master node use:
- Configure moosefs repositories using:
- curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
- curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
- Install required packages for master
- dnf -y install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli
- Configure master server
- cd /etc/mfs
- cp mfsmaster.cfg.sample mfsmaster.cfg
- cp mfsexports.cfg.sample mfsexports.cfg
- Edit /etc/mfs/mfsexport.cfg and append password=<secret> for both root (/) and meta(.) config lines
- Create '/etc/default/moosefs-master' with:
- MFSMASTER_ENABLE=true
- Start and enable moosefs-master service. Also check its running status:
- systemctl start moosefs-master
- systemctl enable moosefs-master
- systemctl status moosefs-master
- Configure CGI server to start on boot by creating '/etc/default/moosefs-cgiserv' with
- MFSCGISERV_ENABLE=true
- Start and enable moosefs-cgiserv service using:
- systemctl start moosefs-cgiserv
- systemctl enable moosefs-cgiserv
- systemctl status moosefs-cgiserv
- Disable and stop firewalld
- systemctl disable firewalld
- systemctl stop firewalld
- Now the CGI can be accessed at http://172.31.1.183:9425/ Most of the values would be blank as there are no other nodes in the setup at the moment.
- We can also run mfscli commands such as:
- mfscli -SIN
- Use 'mfscli -h' to get help on using mfscli command.
Metalogger setup
As explained earlier in case of master failure, the data from metalogger can be used to build a new master. To setup metalogger server use:
- Configure moosefs repositories using:
- curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
- curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
- Install required packages for metalogger using:
- dnf -y install moosefs-metalogger
- Configure metalogger using:
- cd /etc/mfs
- cp mfsmetalogger.cfg.sample mfsmetalogger.cfg
- Configure metalogger to start automatically on boot by creating '/etc/default/moosefs-metalogger' with
- MFSMETALOGGER_ENABLE=true
- Stop and disable firewalld using:
- systemctl stop firewalld
- systemctl disable firewalld
- Start and enable metalogger service using:
- systemctl start moosefs-metalogger
- systemctl enable moosefs-metalogger
- systemctl status moosefs-metalogger
- You can see the metalogger server on servers tab of master web CGI interface
Chunk server setup
- Configure moosefs repositories using:
- curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
- curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
- Install required packages for chunk server
- dnf -y install moosefs-chunkserver
- Configure chunk server using:
- cd /etc/mfs
- cp mfschunkserver.cfg.sample mfschunkserver.cfg
- cp mfshdd.cfg.sample mfshdd.cfg
- It is possible for moosefs to use any folder for storing chunks. In case you want to use entire disks, it is best to format them and mount them appropriately eg /mnt/mfschunks1 etc. Example steps for formatting /dev/sdb and mounting it at /mnt/mfschunks1 are:
- Create a single partition for storing chunks
- parted --align optimal /dev/sdb
- (parted) mklabel gpt
- (parted) mkpart mfschunks1 0% 100%
- (parted) q
- Install xfs utilities
- dnf -y install xfsprogs
- Create required xfs filesystem using:
- mkfs.xfs /dev/sdb1
- If the drives have 4k sector size use 'mkfs.xfs -s size=4k /dev/sdb1' instead
- Create appropriate entires in /etc/fstab to mount the partition on /mnt/mfschunks1 folder
- mkdir /mnt/mfschunks1
- blkid
- #vim /etc/fstab
- #UUID="842950d7-89bd-4272-9532-24773a74d164" /mnt/mfschunks1 xfs defaults 0 0
- mount -a
- df -h
- Create a single partition for storing chunks
- Set correct ownership and permission on folder to be used for mfs chunks
- chown mfs:mfs /mnt/mfschunks1
- chmod 770 /mnt/mfschunks1
- Specify chunkfolder in '/etc/mfs/mfshdd.cfg' as:
- /mnt/mfschunks1
- Configure chunk server to start automatically on boot by creating '/etc/default/moosefs-chunkserver' with
- MFSCHUNKSERVER_ENABLE=true
- Stop and disable firewalld using:
- systemctl stop firewalld
- systemctl disable firewalld
- Start and enable metalogger service using:
- systemctl start moosefs-chunkserver
- systemctl enable moosefs-chunkserver
- systemctl status moosefs-chunkserver
- You can see the chunk servers on servers tab of master web CGI interface
Configure moosefs-client
- Configure moosefs repositories using:
- curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
- curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
- Install required packages for chunk server
- dnf -y install moosefs-client
- Create folder for mounting moosefs data
- mkdir /mnt/mfs
- Mount the folder using:
- mfsmount /mnt/mfs -H mfsmaster -p
- Specify the password given in mfsexport.cfg in mfsmaster
- Check the space of mfs mount
- df -h
- To mount using /etc/fstab instead of using mount command use syntax:
- mfsmount /mnt/primary fuse defaults,mfspassword=<secret> 0 0
- where <secret> is the password given in mfsexport.cfg in mfsmaster
- You can see the client information in mfsmaster CGI web UI.
Refer:
Home > CentOS > CentOS 8.x > CentOS 8.x Distributed Filesystems > CentOS 8.x Moosefs 3.x > CentOS 8.x Moosefs 3.x Initial setup