Migrate sbarjatiya.com VM

<yambe:breadcrumb>New_machine_configuration|New machine configuration</yambe:breadcrumb>

Migrate sbarjatiya.com VM

VM creation on AWS

1. Create a new AWS account
2. Create a CentOS 7.x VM with updates VM is same region and availability zone as previous VM with. Disk space should be at least same as previous VM.
3. Obtain a elastic IP and associate with the VM
4. Add entry in /etc/hosts of current machine with appropriate name for new elastic IP (eg newcommonhosting)
5. Log into older AWS account using private browser
6. Add entry in /etc/hosts of previous VM with appropriate name for previous elastic IP (eg oldcommonhosting)
7. SSH to oldVM
8. Set correct hostname using

   hostname oldcommonhosting

9. Update /etc/hostname with oldcommonhosting name
10. Exit from SSH and reconnect and verify oldcommonhosting name appears
11. SSH to new VM using centos and root may not work
12. Do "sudo su -" on new VM to get root console
13. Install vim

    yum -y install vim

14. Set correct hostname in /etc/hostname
15. Set hostname for current run

    hostname newcommonhosting

16. Edit /root/.ssh/authorized_keys and allow direct root ssh (150x on first line)

    Also copy saurabh@labpc as authorized on new VM root account

    Also copy root@rekallcm1 as authorized on new VM root account

17. Exit from new VM and SSH again as root without using any additional identity apart from saurabh@labpc. Verify newcommonhosting name appears.
18. Fully update the VM to latest packages

    yum -y update --skip-broken

19. Create swap file as mentioned at CentOS 7.x adding swap space using file
20. Configure security group commonhosting-sg with same rules as existing VM. That is access to
    • SSH (22)
    • HTTP (80), HTTPS (443)
    • SMTP (25), SMTPS (465)
    • Custom Alt-web (8080)
    • ICMP echo-request

    from anywhere

21. setenforce 0 on new server
22. edit /etc/sysconfig/selinux and set SELINUX=disabled on new server
23. Use Storing date / time along with commands in history
24. Reboot the new VM

Package installations

1. yum -y install epel-release wget
2. Copy old servers public key as authorized on new server. Run 'ssh-keygen' on old server if there is no existing public key.
3. Create /etc/hosts entry on old server for pointing to new server
4. ssh from oldserver to newserver with name (eg newcommonhosting) and accept the ssh fingerprint of new host
5. rsync /mnt/data1 from old server to new server

   rsync -aHz --delete /mnt/data1/ root@newcommonhosting:/mnt/data1/

   Since this will take time, leave this shell running and open new root shell for previous server

Copy user accounts and home folders

1. Copy user account information to new server

   rsync /etc/{passwd,shadow,group} root@newcommonhosting:

2. Do not close SSH to newcommonhosting till steps complete as in between authentication can stop working and future ssh may not work till fixed
3. Open each of the three files (passwd,shadow,group) and manually copy lines for users such as ecc,sbarjatiya to new files
   1. Also change all auth values from 1000 to 500 in various /etc/pam.d files

      grep 1000 /etc/pam.d/*
      #update all files; :%s/1000/500/gc

4. SSH to new server from a new terminal without closing existing connection and validate it is working
5. Copy other files from oldcommonhosting to newcommonhosting using:

   rsync -aHz /home/ root@newcommonhosting:/home/
   rsync -aHz --exclude ".ssh"  --exclude ".bash_history" /root/ root@newcommonhosting:/root/
   rsync -aHz --delete /etc/postfix/ root@newcommonhosting:/etc/postfix/

6. Run "ls -l /home" in new server and ensure that copied passwd, shadow or group entries work as expected
7. If ssh to new server from old server stops then due to unprotected private key error then use:

   chmod 600 /etc/ssh/*

   on new server to fix the issue

8. Restart postfix on new server

   systemctl restart postfix
   systemctl status postfix

9. Run following on both servers and compare to ensure all things got copied successfully

   du -sh /mnt/data1
   du -sh /home
   getent passwd

Configure web server

1. Install required packages on new server

   yum -y install httpd mod_ssl php-mysql php-pdo php-xml php php-mbstring

2. Update php version to 7.x for latest mediawiki using CentOS 7.x Installing PHP 7.x
3. Copy web server configuration from old server to new

   rsync -vtrp /etc/httpd/{conf,conf.d} root@newcommonhosting:/etc/httpd/

4. If Installing lets-encrypt SSL certificate was used copy /etc/letsencrypt from old server to new. Also copy crontab configuration (crontab -l on old server, crontab -e on new server). Also install python2-certbot-apache package on new server.

   #On old server
   rsync -vaHL /etc/letsencrypt/ root@newcommonhosting:/etc/letsencrypt/
   crontab -l
   
   #On new server
   yum -y install python2-certbot-apache
   crontab -e

5. Start and enable web server on new VM

   systemctl start httpd
   systemctl enable httpd
   systemctl status httpd

Install and configure erlang/yaws

1. Install erlang and yaws on new server

   yum -y install erlang yaws

2. Setup yaws using sbarjatiya user as follows

   su - sbarjatiya
   cd ~/erlang/applications/interpreter; erlc *.erl
   cd ~/erlang/applications/wol_application; erlc *.erl
   cd ~/erlang/erlangcentral.com; erlc *.erl

3. Edit start_yaws.sh and replace old hostname with new hostname
4. Edit start_applications.erl and replace old hostname with new hostname
5. Again compiled edited files

   erlc *.erl

6. Try to start yaws using sbarjatiya user

   ./start_yaws.sh

7. Verify whether yaws is running or not

   yaws --ls

8. exit from sbarjatiya user

Configure MySQL and migrate databases

1. Install Mariadb server, bzip2, sshpass

   yum -y install mariadb-server sshpass bzip2

2. Start and enable mariadb database

   systemctl start mariadb
   systemctl enable mariadb
   systemctl status mariadb

3. Look at '/mnt/data1/plain_folders/documents/public_html/notes_wiki/LocalSettings.php' file for MySQL credentials

   mysql
   > create database notes_wiki;
   > grant all on notes_wiki.* to notes_wiki@localhost identified by '<redacted>';
   > flush privileges;

4. Import database backup

   cd /mnt/data1/plain_folders/documents/public_html
   ./import_notes_database.sh

Configure AWStats, copy old logs

1. Install awstats and related packages

   yum -y install awstats perl-Geo-IP

2. Copy awstats configuration, running data and httpd logs from older server to new server

   rsync -aHz --delete /etc/awstats/ root@newcommonhosting:/etc/awstats/
   rsync -aHz --delete /var/lib/awstats/ root@newcommonhosting:/var/lib/awstats/
   rsync -aHz --delete /var/log/httpd/ root@newcommonhosting:/var/log/httpd/

3. Old steps do not work, need to work on these Configure GeoLocation data for awstats:

   cd /root
   wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
   gunzip GeoLiteCity.dat.gz
   mkdir /usr/local/share/GeoIP
   mv -f GeoLiteCity.dat /usr/local/share/GeoIP
   chmod -R 755 /usr/local/share/GeoIP

4. Temporary new steps for GeoIP

   #On new server
   mkdir /usr/local/share/GeoIP
   
   #On old server
   rsync -vtrp /usr/local/share/GeoIP/GeoLiteCity.dat root@newcommonhosting:/usr/local/share/GeoIP/

5. Restart apache

   systemctl restart httpd
   systemctl status httpd

Make new VM primary by updating DNS

1. Change DNS as follows on godaddy.com:
   • rekallsoftware.com :: @
   • energyconservationclub.in :: @
   • erlangcentral.com :: @
   • pbarjatiya.com :: @
   • sbarjatiya.com :: @
2. Shutdown old VM (Do not release elastic IP yet)
3. Ping above domains and look for new IP. If old IP is shown try

   dig -t any sbarjatiya.com

   or +trace option

4. Check following URLs:
   • http://www.rekallsoftware.com/
   • http://www.sbarjatiya.com/
   • http://www.sbarjatiya.com/notes_wiki/
   • http://www.energyconservationclub.in/
   • http://pbarjatiya.com/
   • http://www.erlangcentral.com/
   • http://www.erlangcentral.com/interpreter/index.yaws
   • http://www.sbarjatiya.com/awstats/awstats.pl?config=www.sbarjatiya.com
5. Send email to saurabh@sbarjatiya.com, saurabh@energyconservationclub.in
6. Release elastic IP from old VM. If required raise support request to AWS to unlock EIP from mail.sbarjatiya.com reverse entry so that it can be released.
7. Fill https://aws.amazon.com/forms/ec2-email-limit-rdns-request?catalog=true&isauthcode=true for new IP with name mail.sbarjatiya.com
8. Update ssh known_hosts keys on rekallcm1 for sbarjatiya.com and www.sbarjatiya.com for both saurabh and root users
9. Take one full backup.

<yambe:breadcrumb>New_machine_configuration|New machine configuration</yambe:breadcrumb>