Difference between revisions of "CentOS 8.x Moosefs 3.x Initial setup"

From Notes_Wiki
(Created page with "<yambe:breadcrumb self="Initial setup">CentOS 8.x Moosefs 3.x|Moosefs 3.x</yambe:breadcrumb> =CentOS 8.x Moosefs 3.x Initial setup= ==Theory== To setup moosefs community edit...")
 
m
Line 207: Line 207:
#:: df -h
#:: df -h
#:</pre>
#:</pre>
# To mount using /etc/fstab instead of using mount command use syntax:
#:<pre>
#::mfsmount /mnt/primary fuse defaults,mfspassword=<secret> 0 0
#:</pre>
#:where <secret> is the password given in mfsexport.cfg in mfsmaster
# You can see the client information in mfsmaster CGI web UI.
# You can see the client information in mfsmaster CGI web UI.



Revision as of 14:30, 12 February 2021

<yambe:breadcrumb self="Initial setup">CentOS 8.x Moosefs 3.x|Moosefs 3.x</yambe:breadcrumb>

CentOS 8.x Moosefs 3.x Initial setup

Theory

To setup moosefs community edition you need following types of servers:

master
This is the master server which coordinates all activities within the distributed filesystem among various nodes. This maintains metadata about files (Which file / chunk is at which node).
metalogger
This server can keep track of changes to metadata. If master goes down we should be able to bring up another master using this server
chunk servers
These nodes store data and contribute to storage space / throughput. Chunk servers can specify a mounted filesystem or device that can be used by moosefs for storing data
clients
These nodes mount the distributed file-system. Multiple clients can mount the filesystem for rw concurrently without any issue.


Initialization

For moosefs to work above nodes (master, metalogger, chunkservers, etc.) should resolve each other by name. That can be done by DNS or by /etc/hosts file. For purpose of this document assume following names and IPs are present in /etc/hosts in all nodes:

172.31.1.183  mfsmaster
172.31.1.184  mfsmetalogger
172.31.1.185  mfschunk1
172.31.1.186  mfschunk2
172.31.1.187  mfschunk3
172.31.1.188  mfsclient1

Here name mfsmaster is critical as it is default in most configuration files. If you change this then please plan to change the name appropriately in most configuration files on most nodes.


Master setup

To setup master node use:

  1. Configure moosefs repositories using:
    curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
    curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
  2. Install required packages for master
    dnf -y install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli
  3. Configure master server
    cd /etc/mfs
    cp mfsmaster.cfg.sample mfsmaster.cfg
    cp mfsexports.cfg.sample mfsexports.cfg
  4. Edit /etc/mfs/mfsexport.cfg and append password=<secret> for both root (/) and meta(.) config lines
  5. Create '/etc/default/moosefs-master' with:
    MFSMASTER_ENABLE=true
  6. Start and enable moosefs-master service. Also check its running status:
    systemctl start moosefs-master
    systemctl enable moosefs-master
    systemctl status moosefs-master
  7. Configure CGI server to start on boot by creating '/etc/default/moosefs-cgiserv' with
    MFSCGISERV_ENABLE=true
  8. Start and enable moosefs-cgiserv service using:
    systemctl start moosefs-cgiserv
    systemctl enable moosefs-cgiserv
    systemctl status moosefs-cgiserv
  9. Disable and stop firewalld
    systemctl disable firewalld
    systemctl stop firewalld
  10. Now the CGI can be accessed at http://172.31.1.183:9425/ Most of the values would be blank as there are no other nodes in the setup at the moment.
  11. We can also run mfscli commands such as:
    mfscli -SIN
    Use 'mfscli -h' to get help on using mfscli command.



Metalogger setup

As explained earlier in case of master failure, the data from metalogger can be used to build a new master. To setup metalogger server use:

  1. Configure moosefs repositories using:
    curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
    curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
  2. Install required packages for metalogger using:
    dnf -y install moosefs-metalogger
  3. Configure metalogger using:
    cd /etc/mfs
    cp mfsmetalogger.cfg.sample mfsmetalogger.cfg
  4. Configure metalogger to start automatically on boot by creating '/etc/default/moosefs-metalogger' with
    MFSMETALOGGER_ENABLE=true
  5. Stop and disable firewalld using:
    systemctl stop firewalld
    systemctl disable firewalld
  6. Start and enable metalogger service using:
    systemctl start moosefs-metalogger
    systemctl enable moosefs-metalogger
    systemctl status moosefs-metalogger
  7. You can see the metalogger server on servers tab of master web CGI interface


Chunk server setup

  1. Configure moosefs repositories using:
    curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
    curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
  2. Install required packages for chunk server
    dnf -y install moosefs-chunkserver
  3. Configure chunk server using:
    cd /etc/mfs
    cp mfschunkserver.cfg.sample mfschunkserver.cfg
    cp mfshdd.cfg.sample mfshdd.cfg
  4. It is possible for moosefs to use any folder for storing chunks. In case you want to use entire disks, it is best to format them and mount them appropriately eg /mnt/mfschunks1 etc. Example steps for formatting /dev/sdb and mounting it at /mnt/mfschunks1 are:
    1. Create a single partition for storing chunks
      parted --align optimal /dev/sdb
      (parted) mklabel gpt
      (parted) mkpart mfschunks1 0% 100%
      (parted) q
    2. Install xfs utilities
      dnf -y install xfsprogs
    3. Create required xfs filesystem using:
      mkfs.xfs /dev/sdb1
      If the drives have 4k sector size use 'mkfs.xfs -s size=4k /dev/sdb1' instead
    4. Create appropriate entires in /etc/fstab to mount the partition on /mnt/mfschunks1 folder
      mkdir /mnt/mfschunks1
      blkid
      #vim /etc/fstab
      #UUID="842950d7-89bd-4272-9532-24773a74d164" /mnt/mfschunks1 xfs defaults 0 0
      mount -a
      df -h
  5. Set correct ownership and permission on folder to be used for mfs chunks
    chown mfs:mfs /mnt/mfschunks1
    chmod 770 /mnt/mfschunks1
  6. Specify chunkfolder in '/etc/mfs/mfshdd.cfg' as:
    /mnt/mfschunks1
  7. Configure chunk server to start automatically on boot by creating '/etc/default/moosefs-chunkserver' with
    MFSCHUNKSERVER_ENABLE=true
  8. Stop and disable firewalld using:
    systemctl stop firewalld
    systemctl disable firewalld
  9. Start and enable metalogger service using:
    systemctl start moosefs-chunkserver
    systemctl enable moosefs-chunkserver
    systemctl status moosefs-chunkserver
  10. You can see the chunk servers on servers tab of master web CGI interface



Configure moosefs-client

  1. Configure moosefs repositories using:
    curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
    curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo
  2. Install required packages for chunk server
    dnf -y install moosefs-client
  3. Create folder for mounting moosefs data
    mkdir /mnt/mfs
  4. Mount the folder using:
    mfsmount /mnt/mfs -H mfsmaster -p
    Specify the password given in mfsexport.cfg in mfsmaster
  5. Check the space of mfs mount
    df -h
  6. To mount using /etc/fstab instead of using mount command use syntax:
    mfsmount /mnt/primary fuse defaults,mfspassword=<secret> 0 0
    where <secret> is the password given in mfsexport.cfg in mfsmaster
  7. You can see the client information in mfsmaster CGI web UI.


Refer:


<yambe:breadcrumb self="Initial setup">CentOS 8.x Moosefs 3.x|Moosefs 3.x</yambe:breadcrumb>