View source for CentOS 7.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities

<yambe:breadcrumb>CentOS_7.x_New_machine_configuration|CentOS 7.x New machine configuration</yambe:breadcrumb>
=CentOS 7.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities=

=Backup before formatting=
Before formatting ensure that there is backup for on external drive 
* /etc
* /home
* /root
* /mnt/data1 appropriate sub-folders
* Last hourly.0 backup of cloud VMs excluding local backups
 

For backup to external disk, use steps similar to:
<pre>
  mount /dev/sd<n>1 /mnt/usb
  mkdir /mnt/usb/2019-08-07-rekallcm1-backup
  rsync -axH /etc/ /mnt/usb/2019-08-07-rekallcm1-backup/etc/
  rsync -axH /home/ /mnt/usb/2019-08-07-rekallcm1-backup/home/
  cd /mnt/data1
  mkdir -p /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1
  rsync -axH plain_folders workspace /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1/
  rsync -axH large_files --exclude=large_files/rsnapshot_backups /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1/
  rsync -axH large_files/rsnapshot_backups/hourly.0 --exclude=hourly.0/rekallcm1 /mnt/usb/2019-08-07-rekallcm1-backup/rsnapshot_backups
</pre>

Ensure that there is enough space in external disk (=df -h=) as required for various folders (=du -sh=).  It is also important to check health of backup disk (=smartctl=) against failed self-test or other pre-failure alert signals.


=OS installation=
Install OS using following options (Assuming 3 disks of 2TB each):
* If necessary validate hardware information eg processor, disks, RAM, etc.  You can look at =smartctl= information of disks for results of last self-tests and any possible pre-failure alerts. 
* Partitioning:
*:; /boot/efi : 1GB - EFI filesytem - RAID 1
*:; / : 50GB - ext4 - RAID 1
*:; swap : 18GB - swap - LVM with Volume group using RAID 5
*:; /mnt/data1 : Remaining - ext4 - LVM with Volume group using RAID 5
* Disable kdump
* Do not apply any security profile
* Server with GUI without Infiband, smart card, FTP related modules
* Enable networking and configure appropriate hostname (Eg rekallcm1)
* Create saurabh user with administrative privileges

Note that OS installation would be very slow while the RAID is building on large 3.8TB /mnt/data1 partitioning.  It would take close to 5-6 hours for the installation to complete when installing through USB stick.


=Initial configuration and package installations=
==Enable NOPASSWD based sudo access==

# Login with saurabh user
# Enter saurabh user password after typing =sudo su -=
# Run '<tt>visudo</tt>' and comment below line
#:<pre>
#::       %wheel  ALL=(ALL)   ALL
#:</pre>
#Uncomment below line
#:<pre>
#::       # %wheel        ALL=(ALL)       NOPASSWD: ALL
#:</pre>
# Exit root terminal and test by typing =sudo su -=.  This time there should not be any password prompt. 



==IP addressing==

# Configure static IP address using Network Manager GUI.  
# Add the same IP in other hosts /etc/hosts files for easy access via name.
# Validate network connectivity
#:<pre>
#::     ping 4.2.2.2
#:</pre>
# Validate DNS resolution
#:<pre>
#::     ping www.google.co.in
#:</pre>
   
Note that this temporary.  We will configure software bridge later for VMs and containers and disable access to interface via Network  Manager (NM_CONTROLLED=no)



==Configure history retention==
Refer [[Storing date / time along with commands in history]]   

After this exit all running shells and reopen new shell.  Validate history is stored with date/time information. 



==Full update==
# Edit /etc/yum.conf and set =keepcache=1=
# Do full update.  This is required so that if any packages that were used before formatting were updated and have updated db/config files, those things can be read properly without version related issues:
#:<pre>
#::     yum -y update --skip-broken
#:</pre>
# Reboot into latest stable kernel

If the RAID partition rebuild is still not finished the full-update would run very slow, same as OS installation.  Check software RAID rebuild status using
<pre>
   cat /proc/mdstat
</pre>


==Disable screen lock==
Start <tt>gnome-control-center</tt> and under Privacy, disable automatic screen-lock



==Disable SELinux==
# setenforce 0
# edit /etc/sysconfig/selinux
#:<pre>
#::     SELINUX=disabled
#:</pre>
This is required as SELinux related errors are very hard to troubleshoot as the issues happen elsewhere apache, image editor, etc. and it is hard to connect that something is not working due to SELinux.  


==Restore backup to /mnt/data1==
# Install byobu for running tasks in background
#:<pre>
#::     yum -y install epel-release
#::     yum -y install byobu
#:</pre>
# Mount external drive and copy files to /mnt/data1 after the raid sync operation has finished (Could take 4-5 hours for 3.8TB):
#:<pre>
#::     mkdir /mnt/usb
#::     mount /dev/sd<n>1 /mnt/usb
#::     byobu
#::
#::     #Inside byobu shell
#::     rsync -axH /mnt/usb/2019-08-06-rekallcm1-backup /mnt/data1/
#::     #Press F6 to exit, Run byobu again to check rsync has completed or not
#:</pre>
# After backup is complete
#:<pre>
#::     umount /mnt/usb
#:</pre>
# Move backup folders to appropriate place '''Be careful with relative vs absolute path while doing below operations'''
#:<pre>
#::     cd /mnt/data1/2019-08-06-rekallcm1-backup
#::     mv /mnt/data1/2019-08-06-rekallcm1-backup/mnt/data1/* /mnt/data1/
#::     rmdir -p mnt/data1
#::
#::     #move rsnapshot_backups inside /mnt/data1/large_files with latest cloud VM backups inside hourly.0 folder
#::
#::     mv home/* /home/*
#::     mv -f home/saurabh/* home/saurabh/.* /home/saurabh/
#::
#::     ls -al home/saurabh  #Ensure empty except local, gnupg, dbus, config, cache 
#::     #For example if firefox was opened as saurabh user before restoring backup then
#::     #rm -rf /home/saurabh/.mozilla
#::     #mv home/saurabh/.mozilla 
#::     
#::     #After home/saurabh is empty
#::     rm -rf home/saurabh
#::     rmdir -p home
#::
#::     ls -al /home/saurabh  #Ensure .emacs, .thunderbird, .mozilla are symlinks
#::
#::     ls -l /home
#::     useradd jitender #UID 1001
#::     useradd achal    #UID 1002
#::     passwd jitender 
#::     passwd achal
#::
#::     mv -f root/* root/.* /root/
#::
#::     ln -s /mnt/data1/plain_folders/documents/ /documents
#::
#::     #exit and ssh again to rekallcm1
#::     #Validate older history from backup is visible
#::     history | less 
#:</pre>



==Install and configure required packages==
===Yum based packages===
<pre>
    yum -y remove bind-chroot
    yum -y install htop   #Colorful command-line top
    yum -y install gimp   #Advanced image editor useful for editing pdf as images
    yum -y install kile  #Latex editor
    yum -y install wireshark wireshark-gnome #Packet capturing
    yum -y install erlang yaws #Programming
    yum -y install fdupes      #Duplicate file detection
    yum -y install dia         #Diagram such as flowcharts or network design editor
    yum -y install libreoffice #Libreoffice
    yum -y install p7zip       #Support for 7zip
    yum -y install kolourpaint #Basic image editor
    yum -y install expect      #For shell automation.  Includes mkpasswd utility
    yum -y install git bzr kdiff3  #For versioning and comparing files
    yum -y install bind bind-utils   #For DNS related tasks
    yum -y install unetbootin liveusb-creator  #For creating live bootable USB from iso files
    yum -y install xfreerdp    #RDP based Remote Desktop
</pre>


===Install and configure owncloud===
# Install owncloud client using [[CentOS 7.x Owncloud desktop client installation]]
# ssh -X saurabh@rekallcm1
# owncloud &
#* Local folder - /mnt/data1/workspace/gbb/owncloud
# Ensure that sync is happening properly
# Resolve all conflicts 
#* Might require deleting older renamed files and restarting owncloud client
# exit from ssh



===Install and configure emacs===
# yum -y install emacs
# Ensure /home/saurabh/.emacs and /home/saurabh/.emacs.d are symlink to appropriate folders of application-data
# Validate things are working by opening to-do.org file 
# Configure nautilus to open .org file via emacs by default



===Install vlc===
Refer [[CentOS 7.x Installing VLC]]


===Configure nautlius===
'''From this point onwards we can work directly on rekallcm1, instead of working remotely'''
# Open nautilus
# Remove /home/saurabh/{Music,Pictures,Videos,Templates}
# Create bookmarks for  
#* /mnt/data1/workspace/gbb/owncloud/saurabh
#* /mnt/data1
#* /mnt/data1/plain_folders/documents/Desktop/rekall
# Configure nautilus to open .org file via emacs by default, if not done already



===Configure firefox===
# Ensure that /home/saurabh/.mozilla is symlink to application-data folder
# Starting firefox and see that various profiles are appearing properly. 
# If required use following to restore .mozilla again from backup:
#:<pre>
#::      mount /mnt/sd<n>1 /mnt/usb
#::      cd /mnt/usb/2019-08-06-rekallcm1-backup/mnt/data1/workspace/application-data/mozilla
#::      rsync -axH --delete ./ /mnt/data1/workspace/application-data/mozilla/
#::      umount /mnt/usb
#:</pre>
# Validate history and stored passwords are working as expected



===Install and configure thunderbird===
# yum -y install thunderbird
# Launch thunderbird and validate that various emails accounts are visible 



===Install anydesk===
# Refer [[CentOS 7.x installing anydesk]]  
# As such below should work:
#:<pre>
#::      yum -y localinstall /mnt/data1/large_files/setup/anydesk/anydesk-5.1.1-1.el7.x86_64.rpm
#:</pre>
# Start anydesk and validate older saved IDs are coming properly



===Configure SSH===
# Validate /home/saurabh/.ssh folder has properly restored backup
# ssh root@sbarjatiya.com 
#* Enter passphrase and enable automatic unlock during login



===Install Google chrome===
# Refer [[CentOS 7.x Installing Google chrome]]
# Start chrome
# Do not make chrome default browser
# Disable submitting of usage information 
# Test by opening www.google.com



===Install flash-plugin===
# Open https://get.adobe.com/flashplayer/
# Choose download of "Yum for Linux"
# As such below should work:
#:<pre>
#::      yum -y localinstall /mnt/data1/large_files/setup/flash/adobe-release-x86_64-1.0-1.noarch.rpm
#:</pre>
# Close all browsers (Esp. firefox)
# yum -y install flash-plugin
# Start firefox and check "about:plugins"


===Configure git===
#Configure various git parameters:
#:<pre>
#::    git config --global user.name "Saurabh Barjatiya"
#::    git config --global user.email "saurabh@sbarjatiya.com"
#::    git config --global color.ui true
#::    git config --global color.status auto
#::    git config --global color.branch auto 
#::    git config --global core.editor vim 
#:</pre>
# Configure to use simple push by default
#:<pre>
#::    git config --global push.default simple
#:</pre>
#:: Refer https://stackoverflow.com/questions/21839651/git-what-is-the-difference-between-push-default-matching-and-simple


===Configure bazaar===
#Configure bazaar parameters:
#:<pre>
#:: bzr whoami "Saurabh Barjatiya <saurabh@sbarjatiya.com>"
#:</pre>


===Enable bind===
# Configure bind for starting automatically:
#:<pre>
#:: systemctl start named
#:: systemctl enable named
#:</pre>
# Change DNS of machine in /etc/sysconfig/network-scripts/ifcfg-br0 file to 127.0.0.1
# Optionally edit /etc/named.conf as follows:
## Update first listen to '<tt>listen-on port 53 { 127.0.0.1; any; };</tt>'
## Comment ''<tt>listen-on-v6 port 53 { ::1; };</tt>' using '<tt>//</tt>' as prefix
## Update allow-query to '<tt>allow-query     { localhost; 10.0.0.0/8; 192.168.0.0/12; 172.16.0.0/16; };</tt>'
# [[CentOS 7.x Disabling IPv6 lookups in bind]]
# Restart named if not done already


=Setup various servers and services=
==Configure software bridge==
# Refer [[Creating bridge interfaces (br0) for virtual hosts to use shared interface]] section "Bridge interfaces in CentOS 7.0"
# Validate that bridge is having ip address and not interface:
#:<pre>
#::     ip addr show
#::     brctl show
#:</pre>


==Install and configure lxc==
# Install lxc by referring [[CentOS 7.x installing lxc]]



==Import various /etc/hosts entries==
# Open file /mnt/data1/2019-08-06-rekallcm1-backup/etc/hosts and copy valid entries and paste them in /etc/hosts file


==Re-add various older VMs==
===Add the VMs via disk or config file===
# Start virt-manager as root user.  Add connection to Qemu/KVM if not present.
#:<pre>
#::       virt-manager
#:</pre>
# Go to File -> New virtual machine
## Choose "Import existing disk image"
## Select VM disk (Eg =/mnt/data1/large_files/vms/tally_vm/tally_vm.qcow2= )
## Select OS type "Linux" and Version "CentOS 7.0"  
## Give required RAM and CPU (Eg 2048, 2 vCPU)
## Ensure that bridge br0 is selected
## Give appropriate VM name (Eg tally_vm)
## Select "Customize configuration before install"
## For "Disk 1" select "Disk Bus" as "IDE" instead of virtio
## For CPU select copy host CPU configuration option
## For Windows XP on NIC choose "e1000" instead of default virtio
## Click "Begin installation" to boot the VM. 
# Download Intel Pro/1000 MT drivers for 32-bit and 64-bit windows from: https://downloadcenter.intel.com/download/18717/Network-Adapter-Drivers-for-Windows-XP-Final-Release?product=50485     As such 32-bit drivers from above link works for Tally_VM.  But for other OS / VMs consider using one of the below as required:
#* Download Linksys E1000 drivers from https://www.driverguide.com/driver/download/Linksys-E1000
#* Download virtio drivers iso from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso
# Copy the downloaded setup to pen drive and make it available to VM using USB redirection.
# Install the setup and Ethernet should get detected.


===Take latest snapshot of VM and delete older snapshots via qemu-img===
# Start virt-manager and take at least one snapshot of all important VMs.
# If there are VM folders in /mnt/data1/large_files/vms which are not useful to be added then delete those folders
# Go to vms folder and list snapshots of existing vm disks:
#:<pre>
#::       cd /mnt/data1/large_files/vms/tally_vm
#::       qemu-img snapshot -l tally_vm.qcow2
#::       qemu-img snapshot -d '<Old-snapshot-name>' tally_vm.qcow2
#:</pre>
#::  '''Be careful and do not delete recent snapshot taken'''
# Boot the VM and validate that it is still booting properly
 

 
==Postfix and logwatch==
# Configure postfix gmail relay using [[Sending email via Gmail relay through postfix]]
# Edit /etc/aliases and add barjatiya.saurabh [at] gmail.com as alias for root.
# Install and configure logwatch



==Setup apache, mariadb including DB restore and php for wikis==
# Replace httpd.conf from configuration folder:
#:<pre>
#::      mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf-old
#::      ln -s /documents/room-documents/documents/general/configuration_files/httpd.conf /etc/httpd/conf/
#:</pre>
# Start and enable mariadb
#:<pre>
#::      systemctl start mariadb
#::      systemctl enable mariadb
#:</pre>
# Check old wiki passwords at following locations:
#* /documents/public_html/notes_wiki/LocalSettings.php
#* /documents/room-documents/documents/databases/mysql/notes_wiki/mediawiki/LocalSettings.php
#* /documents/room-documents/documents/databases/mysql/research_wiki/mediawiki/LocalSettings.php
#* /documents/room-documents/documents/databases/mysql/readme_wiki/mediawiki/LocalSettings.php
# Login as root user (=mysql -u root=) and create other database accounts:
#:<pre>
#::      create database wikidb_notes;
#::      grant all on wikidb_notes.* to wikidb_notes@localhost identified by '<password>';
#::
#::      create database wikidb_res;
#::      grant all on wikidb_res.* to wikidb_res@localhost identified by '<password>';
#::
#::      create database wikidb_readme;
#::      grant all on wikidb_readme.* to wikidb_readme@localhost identified by '<password>';
#::
#::      create database notes_wiki;
#::      grant all on notes_wiki.* to notes_wiki@localhost identified by '<password>';
#::
#::      flush privileges;
#::      \q
#:</pre>
# Restore various database backups using:
#:<pre>
#::        cd /documents/public_html/
#::        bunzip2 -k notes_wiki.sql.bz2
#::        cat notes_wiki.sql | mysql -u notes_wiki -p<password> notes_wiki
#::        rm -f notes_wiki.sql
#::
#::        cd /documents/room-documents/documents/databases/mysql/notes_wiki/
#::        bunzip2 -k wikidb_notes.sql.bz2
#::        cat wikidb_notes.sql | mysql -u wikidb_notes -p<password> wikidb_notes
#::        rm -f wikidb_notes.sql
#::
#::        cd /documents/room-documents/documents/databases/mysql/research_wiki
#::        bunzip2 -k wikidb_res.sql.bz2
#::        cat wikidb_res.sql | mysql -u wikidb_res -p<password> wikidb_res
#::        rm -f wikidb_res.sql
#::
#::        cd /documents/room-documents/documents/databases/mysql/readme_wiki
#::        bunzip2 -k wikidb_readme.sql.bz2
#::        cat wikidb_readme.sql | mysql -u wikidb_readme -p<password> wikidb_readme
#::        rm -f wikidb_readme.sql
#:</pre>
# Install php7 for latest notes_wiki
#:<pre>
#::      yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
#::      yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
#::      yum -y install yum-utils
#::      yum-config-manager --enable remi-php72
#::      yum -y update php
#:</pre>
# Install required packages for php and php-mysql:
#:<pre>
#::      yum -y install php-mysql php php-xml httpd mariadb-server mariadb mod_ssl php-mbstring
#:</pre>
# Restart and enable httpd:
#:<pre>
#::      systemctl restart httpd
#::      systemctl enable httpd
#:</pre>


==Setup postgreSQL databases==
# Install postgreSQL and start DB
#:<pre>
#::      yum -y install postgresql postgresql-server
#::      postgresql-setup initdb
#::      systemctl start postgresql
#::      systemctl enable postgresql
#:</pre>
# Restore postgreSQL configuration file to allow password based logins:
#:<pre>
#::      mv /var/lib/pgsql/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf-old
#::      sudo ln -s /documents/room-documents/documents/general/configuration_files/pg_hba.conf /var/lib/pgsql/data/
#::      sudo chown postgres:postgres /documents/room-documents/documents/general/configuration_files/pg_hba.conf
#::      systemctl restart postgresql
#:</pre>
# Read existing DB password in 
#* /documents/room-documents/public_html/bookworm/global/library/global_common.php
# Use below to create required databases:
#:<pre>
#::        sudo su - postgres
#::        psql
#::
#::        CREATE USER saurabh WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
#::        CREATE DATABASE saurabh WITH OWNER=saurabh;
#::
#::        CREATE USER sen WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
#::        CREATE DATABASE sen WITH OWNER=sen;
#::        \q
#::
#::        exit
#:</pre>
# Restore various database backups using:
#:<pre>
#::      cd /documents/room-documents/documents/databases/pgsql/
#::      psql -U saurabh -d saurabh < saurabh_saurabh.sql
#::    
#::      # cd /documents/room-documents/documents/databases/pgsql/
#::      # bunzip2 -k sen.sql.bz2
#::      # psql -U sen -d sen < sen.sql
#::      # rm -f sen.sql
#:</pre>
# Install php postgresql libraries:
#:<pre>
#::      yum -y install php-pgsql
#::      systemctl restart httpd
#:</pre>
#::  Note that localhost/bookworm does not seems to work with latest php packages



==Configure backups for localhost and AWS VMs==
# Install required packages
#:<pre>
#::      yum -y install rsnapshot sshpass
#:</pre>
# Copy rsnapshot backup file to /etc
#:<pre>
#::      mv /etc/rsnapshot.conf /etc/rsnapshot.conf-old
#::      cd /mnt/data1/2019-08-06-rekallcm1-backup
#::      cp etc/rsnapshot.conf /etc/
#:</pre>
# Create rsnapshot root folder:
#:<pre>
#::      mkdir -p /mnt/data1/large_files/rsnapshot_backups/
#:</pre>
# Edit /etc/rsnapshot.conf and comment backups which are not required or incorrect
# Test backups using
#:<pre>
#::      rsnapshot hourly
#:</pre>
# Edit root crontab using =crontab -e= and use:
#:<pre>
#::      0 0,4,8,12,16,20 * * *       /usr/bin/rsnapshot hourly
#::      45 23 * * *       /usr/bin/rsnapshot daily
#::      30 23 * * 0       /usr/bin/rsnapshot weekly
#::      15 23 1 * *       /usr/bin/rsnapshot monthly
#:</pre>


Also see:
*[[Configuring laptop after Cent-OS re-installation]]
*[[Configure high-end desktop as server for lab experiments]]





<yambe:breadcrumb>CentOS_7.x_New_machine_configuration|CentOS 7.x New machine configuration</yambe:breadcrumb>