Difference between revisions of "CentOS 7.x Zimbra backup and restore"

From Notes_Wiki
(Created page with "<yambe:breadcrumb self="Zimbra backup and restore">CentOS_7.x_Zimbra_configuration|Zimbra configuration</yambe:breadcrumb> =CentOS 7.x Zimbra backup and restore= ==Full backu...")
 
m
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
<yambe:breadcrumb self="Zimbra backup and restore">CentOS_7.x_Zimbra_configuration|Zimbra configuration</yambe:breadcrumb>
[[Main Page|Home]] > [[CentOS]] > [[CentOS 7.x]] > [[CentOS 7.x email configuration|Email configuration]] > [[CentOS 7.x Zimbra configuration|Zimbra configuration]] > [[CentOS 7.x Zimbra backup and restore]]
=CentOS 7.x Zimbra backup and restore=


==Full backup and restore of /opt/zimbra==
==Full backup of Zimbra along with DB and LDAP==
Zimbra keeps all its required files under /opt/zimbra.  A full backup of /opt/zimbra (*Esp when zimbra is not running*) contains consistent backup of email server configuration, user preferences, user emails, etc.  A Linux utility such as rsnapshot can be used for full backups at required intervals.   
Zimbra keeps all its required files under /opt/zimbra.  A full backup of /opt/zimbra (*Esp when zimbra is not running*) contains consistent backup of email server configuration, user preferences, user emails, etc.  A Linux utility such as rsnapshot can be used for full backups at required intervals.  For full backup using rsnapshot use:
 
# Create folder for taking ldap backup using:
#:<pre>
#::  mkdir /opt/zimbra-ldap-backup
#:: chown -R zimbra:zimbra /opt/zimbra-ldap-backup
#:</pre>
# Create ldap_backup_script on zimbra server at <tt>/opt/take_zimbra_ldap_backup.sh</tt> with following contents:
#:<pre>
#::    #!/bin/bash
#::
#::    rm -f /opt/zimbra-ldap-backup/*
#::    su - zimbra -c "/opt/zimbra/libexec/zmslapcat -c /opt/zimbra-ldap-backup/"
#::    su - zimbra -c "/opt/zimbra/libexec/zmslapcat /opt/zimbra-ldap-backup/"
#:</pre>
# chmod +x /opt/take_zimbra_ldap_backup.sh
# Create folder for taking mysql backup using:
#:<pre>
#:: mkdir /opt/zimbra-mysql-backup
#:: chown -R zimbra:zimbra /opt/zimbra-mysql-backup
#:</pre>
# Create '<tt>/opt/take_zimbra_mysql_backup.sh</tt>' with following contents:
#:<pre>
#:: #!/bin/bash
#:: export backup_dir="/opt/zimbra-mysql-backup"
#::su - zimbra -c '
#::source /opt/zimbra/bin/zmshutil;
#::zmsetvars;
#::/opt/zimbra/common/bin/mysqldump --user=root --password=$mysql_root_password --socket=$mysql_socket --all-databases --single-transaction --flush-logs;
#::' | gzip > $backup_dir/mysqldump.sql.gz
#:</pre>
# chmod +x /opt/take_zimbra_mysql_backup.sh
# Configure rsnapshot as follows:
#:<pre>
#::    backup_script  /usr/bin/ssh root@172.26.2.54 '/opt/take_zimbra_ldap_backup.sh'        mail.sbarjatiya.com/.ignore1/
#::    backup_script  /usr/bin/ssh root@172.26.2.54 '/opt/take_zimbra_mysql_backup.sh'        mail.sbarjatiya.com/.ignore2/
#::    backup  root@172.26.2.54:/opt/          mail.gbb.co.in/        +rsync_long_args=--exclude=/opt/zimbra/data/ldap/ --sparse -aHz
#:</pre>
#:: Note following
#* Use tabs appropriately instead of spaces in above config for rsnapshot.conf
#* Note that we need to call backup_script before taking backup of complete /opt. 
#* Also note '''use of --exclude=/opt/zimbra/data/ldap which is very crucial''' for backup to work as there is sparse file in /opt/zimbra/data/ldap/mdb/db which is quite big (80GB sparse).
 
 
Refer:
* https://serverfault.com/questions/680942/backing-up-zimbra-8-0-x-data-mdb-issues
* https://rizvir.com/articles/zimbra-8-production-setup/
 
 
==Restoring full backup==


Full restoration is also straightforward using:
Full restoration is also straightforward using:
Line 16: Line 64:


Refer:  
Refer:  
* https://www.sbarjatiya.com/notes_wiki/index.php/Rsnapshot
* [[Rsnapshot]]
* https://www.sbarjatiya.com/notes_wiki/index.php/Take_backup_of_zimbra_server
* [[Take_backup_of_zimbra_server]]




Line 25: Line 73:
   
   
The backup suggested above may not be consistent unless all services are stoppped.  To stop all services before backup and to start them again after backup, we need the backup window to be small.  Full backup on the other hand (Based on total data) might take considerable time.  Thus, there is trade-off between consistent backup and down-time.  However, at any cost at least one weekly full backup after shutting down all services should be taken.
The backup suggested above may not be consistent unless all services are stoppped.  To stop all services before backup and to start them again after backup, we need the backup window to be small.  Full backup on the other hand (Based on total data) might take considerable time.  Thus, there is trade-off between consistent backup and down-time.  However, at any cost at least one weekly full backup after shutting down all services should be taken.
===Using full-backup and restore to get access to particular account or email===
As mentioned above the limitation of full-backup is that even for restoration of a single account or email we need to restore full server.  In case that is required we can use following steps:
# If the zimbra production setup is on a VM, we can clone and get another similar VM.
## For on-prem virtualized VMs, please clone the VM at hypervisor level, boot it and change the NIC IP to avoid IP conflict between clone and original production VM.
## For cloud VMs also create template / image and use it to launch another similar VM.  Do this without rebooting original machine to reduce / prevent downtime.
# Once a similar machine is available due to cloning or due to installation of '''exact''' same version of zimbra on '''exactly''' same OS, then we can work on restoring from backup
## First login as zimbra user and stop services '''as zimbra user'''
##:<pre>
##:: zmcontrol stop
##:</pre>
## Then choose appropriate backup to be restored.  If the backup is via rsnapshot and we have already identified correct backup to be restored, we can use following:
##:<pre>
##:: cd <appropriate opt folder from backup>
##:: rsync -aHz --delete ./ root@3.108.10.133:/opt/
##:</pre>
##:: '''Caution: Ensure that files are being restored to clone VM and not to original production VM'''
## Set correct hostname on server based on OS:
##:<pre>
##:: hostnamectl set-hostname <mail-server-fqdn>
##:</pre>
## Ensure that /etc/hosts file has correct mapping between new VMs private IP and FQDN
##:<pre>
##:: cat /etc/hosts
##:</pre>
##: Edit the file and put correct private IP of new restored VM against FQDN
## As per the backup steps we are excluding folder /opt/zimbra/data/ldap/ from backup and taking LDAP backup separately.  So we need to restore the ldap backup using following steps:
### Create ldap related folder '''as root user'''
###:<pre>
###:: mkdir -p /opt/zimbra/data/ldap/mdb/db /opt/zimbra/data/ldap/mdb/logs
###:: mkdir -p /opt/zimbra/data/ldap/config/
###:: chown -R zimbra:zimbra /opt/zimbra/data/ldap
###:: /opt/zimbra/libexec/zmfixperms -verbose
###:</pre>
### Restore zimbra ldap config and ldap data '''as zimbra user'''
###:<pre>
###:: slapadd -q -n 0 -F /opt/zimbra/data/ldap/config -cv -l /opt/zimbra-ldap-backup/ldap-config.bak
###:: slapadd -q -b "" -F /opt/zimbra/data/ldap/config -cv -l /opt/zimbra-ldap-backup/ldap.bak
###:</pre>
# Reboot the machine '''as root user'''
#:<pre>
#:: shutdown -r now
#:</pre>
# Check if all zimbra services start properly '''as zimbra user'''
#:<pre>
#:: zmcontrol status
#:</pre>
# Create folder for taking specific user data backup '''as root user'''
#:<pre>
#:: mkdir /opt/user-data
#:: chown -R zimbra:zimbra /opt/user-data
#:</pre>
# Go to /opt/user-data and take user data backup as .tgz file  '''as zimbra user'''
#:<pre>
#:: zmmailbox -z -m <email-address> getRestURL "//?fmt=tgz" > user-email-data.tar.gz
#:</pre>
# Once the data is backed up we can copy it to running production email server at same path (/opt/user-data) and restore using: '''as zimbra user'''
#:<pre>
#:: cd /opt/user-data
#:: zmprov ca <email-address> <account-password> displayName <email-user-full-name>
#:: zmmailbox -z -m <email-address> postRestURL “//?fmt=tgz&resolve=reset” user-email-data.tar.gz
#:</pre>
Refer:
* https://bobcares.com/blog/restore-zimbra-ldap-database-from-backup/
* https://forums.zimbra.org/viewtopic.php?t=45736
* http://docs.zimbra.com/docs/ne/4.0.5/administration_guide/Managing_Accounts.9.1.html




Line 108: Line 226:




'''There is older article on this at [[Take backup of zimbra server]]'''




<yambe:breadcrumb self="Zimbra backup and restore">CentOS_7.x_Zimbra_configuration|Zimbra configuration</yambe:breadcrumb>
[[Main Page|Home]] > [[CentOS]] > [[CentOS 7.x]] > [[CentOS 7.x email configuration|Email configuration]] > [[CentOS 7.x Zimbra configuration|Zimbra configuration]] > [[CentOS 7.x Zimbra backup and restore]]

Latest revision as of 15:54, 28 August 2022

Home > CentOS > CentOS 7.x > Email configuration > Zimbra configuration > CentOS 7.x Zimbra backup and restore

Full backup of Zimbra along with DB and LDAP

Zimbra keeps all its required files under /opt/zimbra. A full backup of /opt/zimbra (*Esp when zimbra is not running*) contains consistent backup of email server configuration, user preferences, user emails, etc. A Linux utility such as rsnapshot can be used for full backups at required intervals. For full backup using rsnapshot use:

  1. Create folder for taking ldap backup using:
    mkdir /opt/zimbra-ldap-backup
    chown -R zimbra:zimbra /opt/zimbra-ldap-backup
  2. Create ldap_backup_script on zimbra server at /opt/take_zimbra_ldap_backup.sh with following contents:
    #!/bin/bash
    rm -f /opt/zimbra-ldap-backup/*
    su - zimbra -c "/opt/zimbra/libexec/zmslapcat -c /opt/zimbra-ldap-backup/"
    su - zimbra -c "/opt/zimbra/libexec/zmslapcat /opt/zimbra-ldap-backup/"
  3. chmod +x /opt/take_zimbra_ldap_backup.sh
  4. Create folder for taking mysql backup using:
    mkdir /opt/zimbra-mysql-backup
    chown -R zimbra:zimbra /opt/zimbra-mysql-backup
  5. Create '/opt/take_zimbra_mysql_backup.sh' with following contents:
    #!/bin/bash
    export backup_dir="/opt/zimbra-mysql-backup"
    su - zimbra -c '
    source /opt/zimbra/bin/zmshutil;
    zmsetvars;
    /opt/zimbra/common/bin/mysqldump --user=root --password=$mysql_root_password --socket=$mysql_socket --all-databases --single-transaction --flush-logs;
    ' | gzip > $backup_dir/mysqldump.sql.gz
  6. chmod +x /opt/take_zimbra_mysql_backup.sh
  7. Configure rsnapshot as follows:
    backup_script /usr/bin/ssh root@172.26.2.54 '/opt/take_zimbra_ldap_backup.sh' mail.sbarjatiya.com/.ignore1/
    backup_script /usr/bin/ssh root@172.26.2.54 '/opt/take_zimbra_mysql_backup.sh' mail.sbarjatiya.com/.ignore2/
    backup root@172.26.2.54:/opt/ mail.gbb.co.in/ +rsync_long_args=--exclude=/opt/zimbra/data/ldap/ --sparse -aHz
    Note following
    • Use tabs appropriately instead of spaces in above config for rsnapshot.conf
    • Note that we need to call backup_script before taking backup of complete /opt.
    • Also note use of --exclude=/opt/zimbra/data/ldap which is very crucial for backup to work as there is sparse file in /opt/zimbra/data/ldap/mdb/db which is quite big (80GB sparse).


Refer:


Restoring full backup

Full restoration is also straightforward using:

  1. Stop zimbra services.
  2. Rename /opt/zimbra to /opt/zimbra-old if there is enough storage space. Otherwise delete /opt/zimbra
  3. Restore the backup to /opt/zimbra.
  4. If required fix permissions using
    chown -R zimbra:zimbra /opt/zimbra
    /opt/zimbra/libexec/zmfixperms

Refer:


Limitations of full backup

The most critical limitation of full backup is that there is no partial restore option. Either we should restore fully on a second machine and access required backup emails/accounts or we need to restore fully on production server. Ideally backups should allow restoring individual accounts or emails. This is not possible using the backup process which takes backup of /opt/zimbra as suggested above.

The backup suggested above may not be consistent unless all services are stoppped. To stop all services before backup and to start them again after backup, we need the backup window to be small. Full backup on the other hand (Based on total data) might take considerable time. Thus, there is trade-off between consistent backup and down-time. However, at any cost at least one weekly full backup after shutting down all services should be taken.


Using full-backup and restore to get access to particular account or email

As mentioned above the limitation of full-backup is that even for restoration of a single account or email we need to restore full server. In case that is required we can use following steps:

  1. If the zimbra production setup is on a VM, we can clone and get another similar VM.
    1. For on-prem virtualized VMs, please clone the VM at hypervisor level, boot it and change the NIC IP to avoid IP conflict between clone and original production VM.
    2. For cloud VMs also create template / image and use it to launch another similar VM. Do this without rebooting original machine to reduce / prevent downtime.
  2. Once a similar machine is available due to cloning or due to installation of exact same version of zimbra on exactly same OS, then we can work on restoring from backup
    1. First login as zimbra user and stop services as zimbra user
      zmcontrol stop
    2. Then choose appropriate backup to be restored. If the backup is via rsnapshot and we have already identified correct backup to be restored, we can use following:
      cd <appropriate opt folder from backup>
      rsync -aHz --delete ./ root@3.108.10.133:/opt/
      Caution: Ensure that files are being restored to clone VM and not to original production VM
    3. Set correct hostname on server based on OS:
      hostnamectl set-hostname <mail-server-fqdn>
    4. Ensure that /etc/hosts file has correct mapping between new VMs private IP and FQDN
      cat /etc/hosts
      Edit the file and put correct private IP of new restored VM against FQDN
    5. As per the backup steps we are excluding folder /opt/zimbra/data/ldap/ from backup and taking LDAP backup separately. So we need to restore the ldap backup using following steps:
      1. Create ldap related folder as root user
        mkdir -p /opt/zimbra/data/ldap/mdb/db /opt/zimbra/data/ldap/mdb/logs
        mkdir -p /opt/zimbra/data/ldap/config/
        chown -R zimbra:zimbra /opt/zimbra/data/ldap
        /opt/zimbra/libexec/zmfixperms -verbose
      2. Restore zimbra ldap config and ldap data as zimbra user
        slapadd -q -n 0 -F /opt/zimbra/data/ldap/config -cv -l /opt/zimbra-ldap-backup/ldap-config.bak
        slapadd -q -b "" -F /opt/zimbra/data/ldap/config -cv -l /opt/zimbra-ldap-backup/ldap.bak
  3. Reboot the machine as root user
    shutdown -r now
  4. Check if all zimbra services start properly as zimbra user
    zmcontrol status
  5. Create folder for taking specific user data backup as root user
    mkdir /opt/user-data
    chown -R zimbra:zimbra /opt/user-data
  6. Go to /opt/user-data and take user data backup as .tgz file as zimbra user
    zmmailbox -z -m <email-address> getRestURL "//?fmt=tgz" > user-email-data.tar.gz
  7. Once the data is backed up we can copy it to running production email server at same path (/opt/user-data) and restore using: as zimbra user
    cd /opt/user-data
    zmprov ca <email-address> <account-password> displayName <email-user-full-name>
    zmmailbox -z -m <email-address> postRestURL “//?fmt=tgz&resolve=reset” user-email-data.tar.gz


Refer:



Backup and restore of individual accounts

The below is not tested on production extensively, so use with caution.

Backup of individual account is possible. A full backup of account which includes mail, contacts, calendar, tasks, briefcase, etc. using scripts available at https://github.com/garithd/zimbra-permailbox-backup/wiki/Installation The script assumes only one backup per day.

To setup zimbra server to use above scripts use following steps:

  1. Install required software using:
    yum -y install git python-ldap python-requests python-setuptools
  2. Setup Backup scripts using:
    cd /root
    git clone https://github.com/garithd/zimbra-permailbox-backup.git
    cd zimbra-permailbox-backup
    cp zimbra_permailbox_backup.conf /etc
    cp zimbra_permailbox_backup.py /usr/local/bin/
    chmod +x /usr/local/bin/zimbra_permailbox_backup.py
    cd /root
    git clone https://github.com/djmitche/pynsca.git
    cd pynsca
    python setup.py install
  3. Edit /etc/zimbra_permailbox_backup.conf as follows:
    [auths]
    #Find using 'zmlocalconfig -s | grep ldap_root_password' as zimbra user
    mail.zimbra.sbarjatiya.com: <redacted>
    [zimbraauths]
    #Zimbra admin user password
    mail.zimbra.sbarjatiya.com: <redacted>
    [nagioshosts]
    #Not using so far, so no nagios configuration
    [backupdir]
    #Backup destination, should have enough space
    mailbackuptopdir: /mnt/backups/
  4. Optionally, make sure /mnt/backups exists with access to zimbra user
    mkdir /mnt/backups
    chown -R zimbra:zimbra /mnt/backups
    As such backups seem to happen directly as root, so this is not necessary
  5. Test full backup using:
    /usr/local/bin/zimbra_permailbox_backup.py -b all -s mail.zimbra.sbarjatiya.com
  6. List all backups of user1 using:
    /usr/local/bin/zimbra_permailbox_backup.py -r 'user1@zimbra.sbarjatiya.com' -d list -s mail.zimbra.sbarjatiya.com
    Example output is:
    [root@mail ~]# /usr/local/bin/zimbra_permailbox_backup.py -r 'user1@zimbra.sbarjatiya.com' -d list -s mail.zimbra.sbarjatiya.com
    Backup(s) for account: user1@zimbra.sbarjatiya.com in /mnt/backups/mail.zimbra.sbarjatiya.com/mailboxes/
    Date: 20190214, Size: 5.8 KB
  7. To restore backup of given date use:
    /usr/local/bin/zimbra_permailbox_backup.py -r 'user1@zimbra.sbarjatiya.com' -t 'user1@zimbra.sbarjatiya.com' -d 20190214 -s mail.zimbra.sbarjatiya.com
    Example restore output is:
    [root@mail ~]# /usr/local/bin/zimbra_permailbox_backup.py -r 'user1@zimbra.sbarjatiya.com' -t 'user1@zimbra.sbarjatiya.com' -d 20190214 -s mail.zimbra.sbarjatiya.com
    Running a restore to account user1@zimbra.sbarjatiya.com on mail.zimbra.sbarjatiya.com using backup /mnt/backups/mail.zimbra.sbarjatiya.com/mailboxes/20190214/user1@zimbra.sbarjatiya.com.tgz
    Restore complete
    Note that restore account -t need not be same as source account -r. Restoration can be done to a different account also. However, the account has to be created before restore.


Refer:



There is older article on this at Take backup of zimbra server


Home > CentOS > CentOS 7.x > Email configuration > Zimbra configuration > CentOS 7.x Zimbra backup and restore