Migrate sbarjatiya.com VM

From Notes_Wiki
Revision as of 15:36, 12 September 2019 by Saurabh (talk | contribs)

<yambe:breadcrumb>New_machine_configuration|New machine configuration</yambe:breadcrumb>

Migrate sbarjatiya.com VM

VM creation on AWS

  1. Create a new AWS account
  2. Create a CentOS 7.x VM with updates VM is same region and availability zone as previous VM with. Disk space should be at least same as previous VM.
  3. Obtain a elastic IP and associate with the VM
  4. Add entry in /etc/hosts of current machine with appropriate name for new elastic IP (eg newcommonhosting)
  5. Log into older AWS account using private browser
  6. Add entry in /etc/hosts of previous VM with appropriate name for previous elastic IP (eg oldcommonhosting)
  7. SSH to oldVM
  8. Set correct hostname using
    hostname oldcommonhosting
  9. Update /etc/hostname with oldcommonhosting name
  10. Exit from SSH and reconnect and verify oldcommonhosting name appears
  11. SSH to new VM using centos and root may not work
  12. Do "sudo su -" on new VM to get root console
  13. Install vim
    yum -y install vim
  14. Set correct hostname in /etc/hostname
  15. Set hostname for current run
    hostname newcommonhosting
  16. Edit /root/.ssh/authorized_keys and allow direct root ssh (150x on first line)
    Also copy saurabh@labpc as authorized on new VM root account
  17. Exit from new VM and SSH again as root without using any additional identity apart from saurabh@labpc. Verify newcommonhosting name appears.
  18. Fully update the VM to latest packages
    yum -y update --skip-broken
  19. Create swap file as mentioned at CentOS 7.x adding swap space using file
  20. Configure security group commonhosting-sg with same rules as existing VM. That is access to
    • SSH (22)
    • HTTP (80), HTTPS (443)
    • SMTP (25), SMTPS (465)
    • Custom Alt-web (8080)
    • ICMP echo-request
    from anywhere
  21. setenforce 0 on new server
  22. edit /etc/sysconfig/selinux and set SELINUX=disabled on new server
  23. Use Storing date / time along with commands in history
  24. Reboot the new VM


Package installations

  1. yum -y install epel-release wget
  2. Copy old servers public key as authorized on new server. Run 'ssh-keygen' on old server if there is no existing public key.
  3. Create /etc/hosts entry on old server for pointing to new server
  4. ssh from oldserver to newserver with name (eg newcommonhosting) and accept the ssh fingerprint of new host
  5. rsync /mnt/data1 from old server to new server
    rsync -aHz --delete /mnt/data1/ root@newcommonhosting:/mnt/data1/
    Since this will take time, leave this shell running and open new root shell for previous server


Copy user accounts and home folders

  1. Copy user account information to new server
    rsync /etc/{passwd,shadow,group} root@newcommonhosting:
  2. Do not close SSH to newcommonhosting till steps complete as in between authentication can stop working and future ssh may not work till fixed
  3. Open each of the three files (passwd,shadow,group) and manually copy lines for users such as ecc,sbarjatiya to new files
    1. Also change all auth values from 1000 to 500 in various /etc/pam.d files
      grep 1000 /etc/pam.d/*
      #update all files; :%s/1000/500/gc
  4. SSH to new server from a new terminal without closing existing connection and validate it is working
  5. Copy other files from oldcommonhosting to newcommonhosting using:
    rsync -aHz /home/ root@newcommonhosting:/home/
    rsync -aHz --exclude ".ssh" --exclude ".bash_history" /root/ root@newcommonhosting:/root/
    rsync -aHz --delete /etc/postfix/ root@newcommonhosting:/etc/postfix/
  6. Run "ls -l /home" in new server and ensure that copied passwd, shadow or group entries work as expected
  7. If ssh to new server from old server stops then due to unprotected private key error then use:
    chmod 600 /etc/ssh/*
    on new server to fix the issue
  8. Restart postfix on new server
    systemctl restart postfix
    systemctl status postfix
  9. Run following on both servers and compare to ensure all things got copied successfully
    du -sh /mnt/data1
    du -sh /home
    getent passwd


Configure web server

  1. Install required packages on new server
    yum -y install httpd mod_ssl php-mysql php-pdo php-xml php php-mbstring
  2. Update php version to 7.x for latest mediawiki using CentOS 7.x Installing PHP 7.x
  3. Copy web server configuration from old server to new
    rsync -vtrp /etc/httpd/{conf,conf.d} root@newcommonhosting:/etc/httpd/
  4. If Installing lets-encrypt SSL certificate was used copy /etc/letsencrypt from old server to new. Also copy crontab configuration (crontab -l on old server, crontab -e on new server). Also install python2-certbot-apache package on new server.
    #On old server
    rsync -vaHL /etc/letsencrypt/ root@newcommonhosting:/etc/letsencrypt/
    crontab -l
    #On new server
    yum -y install python2-certbot-apache
    crontab -e
  5. Start and enable web server on new VM
    systemctl start httpd
    systemctl enable httpd
    systemctl status httpd


Install and configure erlang/yaws

  1. Install erlang and yaws on new server
    yum -y install erlang yaws
  2. Setup yaws using sbarjatiya user as follows
    su - sbarjatiya
    cd ~/erlang/applications/interpreter; erlc *.erl
    cd ~/erlang/applications/wol_application; erlc *.erl
    cd ~/erlang/erlangcentral.com; erlc *.erl
  3. Edit start_yaws.sh and replace old hostname with new hostname
  4. Edit start_applications.erl and replace old hostname with new hostname
  5. Again compiled edited files
    erlc *.erl
  6. Try to start yaws using sbarjatiya user
    ./start_yaws.sh
  7. Verify whether yaws is running or not
    yaws --ls
  8. exit from sbarjatiya user


Configure MySQL and migrate databases

  1. Install Mariadb server, bzip2, sshpass
    yum -y install mariadb-server sshpass bzip2
  2. Start and enable mariadb database
    systemctl start mariadb
    systemctl enable mariadb
    systemctl status mariadb
  3. Look at '/mnt/data1/plain_folders/documents/public_html/notes_wiki/LocalSettings.php' file for MySQL credentials
    mysql
    > create database notes_wiki;
    > grant all on notes_wiki.* to notes_wiki@localhost identified by '<redacted>';
    > flush privileges;
  4. Import database backup
    cd /mnt/data1/plain_folders/documents/public_html
    ./import_notes_database.sh


Configure AWStats, copy old logs

  1. Install awstats and related packages
    yum -y install awstats perl-Geo-IP
  2. Copy awstats configuration, running data and httpd logs from older server to new server
    rsync -aHz --delete /etc/awstats/ root@newcommonhosting:/etc/awstats/
    rsync -aHz --delete /var/lib/awstats/ root@newcommonhosting:/var/lib/awstats/
    rsync -aHz --delete /var/log/httpd/ root@newcommonhosting:/var/log/httpd/
  3. Old steps do not work, need to work on these Configure GeoLocation data for awstats:
    cd /root
    wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
    gunzip GeoLiteCity.dat.gz
    mkdir /usr/local/share/GeoIP
    mv -f GeoLiteCity.dat /usr/local/share/GeoIP
    chmod -R 755 /usr/local/share/GeoIP
  4. Temporary new steps for GeoIP
    #On new server
    mkdir /usr/local/share/GeoIP
    #On old server
    rsync -vtrp /usr/local/share/GeoIP/GeoLiteCity.dat root@newcommonhosting:/usr/local/share/GeoIP/
  5. Restart apache
    systemctl restart httpd
    systemctl status httpd


Make new VM primary by updating DNS

  1. Change DNS as follows on godaddy.com:
    • rekallsoftware.com :: @
    • energyconservationclub.in :: @
    • erlangcentral.com :: @
    • pbarjatiya.com :: @
    • sbarjatiya.com :: @
  2. Shutdown old VM (Do not release elastic IP yet)
  3. Ping above domains and look for new IP. If old IP is shown try
    dig -t any sbarjatiya.com
    or +trace option
  4. Check following URLs:
  5. Send email to saurabh@sbarjatiya.com, saurabh@energyconservationclub.in
  6. Release elastic IP from old VM. If required raise support request to AWS to unlock EIP from mail.sbarjatiya.com reverse entry so that it can be released.
  7. Fill https://aws.amazon.com/forms/ec2-email-limit-rdns-request?catalog=true&isauthcode=true for new IP with name mail.sbarjatiya.com
  8. Update ssh known_hosts keys on rekallcm1 for sbarjatiya.com and www.sbarjatiya.com.
  9. Take one full backup.


<yambe:breadcrumb>New_machine_configuration|New machine configuration</yambe:breadcrumb>