CentOS 8.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities

From Notes_Wiki
Revision as of 10:07, 25 February 2020 by Saurabh (talk | contribs)

<yambe:breadcrumb self="Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities">CentOS 8.x New machine configuration|New machine configuration</yambe:breadcrumb>

CentOS 8.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities

Older CentOS 7.x article is present at CentOS 7.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities

This is case of complete data loss on rekallcm1 and fresh setup of rekallcm1 using files from laptop(barjatiyarklp)

OS installation

  1. Create a live USB with CentOS 8.x installer
  2. Boot and select server with GUI while skipping:
    • Guest Agents
    • Infiband support
    • .NET Core Development
    • Smart card support
  3. Disable kdump
  4. Configure 192.168.1.3/24 with 192.168.1.251 as gateway and rekallcm1 as hostname
  5. Install OS on software RAID on 3 * 2 TB disks as follows:
    /
    ext4 - RAID 1 - 50 GB
    /boot/efi
    EFI system partition - RAID 1 - 1 GB
    swap
    swap - RAID 5 - 18 GB
    /mnt/data1
    ext4 - RAID 5 - remaining


Install Owncloud Desktop client

CentOS 8.x Owncloud linux desktop client

Sudo configuration

Run 'visudo' and append

   saurabh ALL=(ALL)  NOPASSWD:ALL
   achal ALL=(ALL)  NOPASSWD:ALL
   jitender ALL=(ALL)  NOPASSWD:ALL


Create ext4 partition on raid5 device and mount on /mnt/data1

Use:

   mkfs.ext4 /dev/mapper/centos_rekallcm1-mnt_data1
   mkdir -p /mnt/data1
   blkid  #Note UUID for /dev/mapper/centos_rekallcm1-mnt_data1
   vim /etc/fstab #and append
     UUID="<from-above-command>"  /mnt/data1         ext4    defaults        1 2
   mount -a 


Copy data from laptop (barjatiyarklp) to rekallcm1

   cd /mnt/data1
   rsync -aHz --delete root@<laptop-ip>:/mnt/data1/ ./

   mkdir -p large_files
   mv vms large_files

   mkdir -p large_files/operating_systems
   cd large_files/operating_systems
   rsync -vaHz --inplace root@<laptop-ip>:/mnt/ddrive/operating_systems/ ./

   #Copy /mnt/data1/personal from various backup sources


Configure history retention


Disable screen lock

  1. Start gnome-control-center and under Privacy, disable automatic screen-lock
  2. Under "Power savings" set screen going blank to never


Disable SELinux

  1. setenforce 0
  2. edit /etc/sysconfig/selinux
    SELINUX=disabled


Yum configuration and package installation

  1. Configure keep-cache for yum
    • Edit /etc/yum.conf and set =keepcache=1=
      *Note cache goes to /var/cache/dnf*
  2. Install required packages
    yum -y install epel-release #EPEL repository
    yum -y install htop #Colorful command-line top
    yum -y install gimp #Advanced image editor useful for editing pdf as images
    yum -y install wireshark #Packet capturing
    yum -y install erlang #Programming
    yum -y install fdupes #Duplicate file detection
    yum -y install libreoffice-calc libreoffice-writer libreoffice-impress #Libreoffice
    yum -y install p7zip #Support for 7zip
    yum -y install expect #For shell automation. Includes mkpasswd utility
    yum -y install git #For versioning
    yum -y install bind bind-utils #For DNS related tasks* CentOS 8.x barjatiyarklp
    yum -y install texlive
    yum -y install freerdp #Windows remote desktop
    yum -y install texlive #For latex
    yum -y install thunderbird #For email
    yum -y install byobu #To run shell commands in background
    yum -y install openvpn #For vpn connectivity
    yum -y install ntfs-3g #NTFS file-system support
  3. Packages not found in CentOS 8 at time of writing
    yum -y install yaws #Erlang web server
    yum -y install kile #Latex editor
    yum -y install bzr kdiff3 #For versioning and comparing files
    yum -y install kolourpaint #Basic image editor
    yum -y install dia #Diagram such as flowcharts or network design editor
    yum -y install unetbootin liveusb-creator #For creating live bootable USB from iso files
  4. Full update system
    yum -y update --skip-broken


Install and configure emacs

  1. yum -y install emacs
  2. Ensure /home/saurabh/.emacs and /home/saurabh/.emacs.d are symlink to appropriate folders of application-data
  3. Validate things are working by opening to-do.org file
  4. Configure nautilus to open .org file via emacs by default


Install vlc

  1. yum -y install https://download1.rpmfusion.org/free/el/rpmfusion-free-release-8.noarch.rpm
  2. yum -y install vlc

Refer:


Download and configure anydesk

  1. Go to https://anydesk.com/en/downloads/linux and Download Cent OS 8 (64-bit) anydesk rpm
  2. yum -y install redhat-lsb-core
  3. yum -y localinstall <downloaded-rpm>
  4. Set unattended password for anydesk
    • Open anydesk as normal user (Saurabh)
    • Unlock security settings by entering root password
    • Ensure interactive access is set to "Allow always"
    • Now try to set password for unattended access


Install chrome

  1. Refer CentOS 7.x Installing Google chrome
  2. Start chrome
  3. Do not make chrome default browser and Disable submitting of usage information
  4. Test by opening www.google.com


Configure git

  1. Configure various git parameters:
    git config --global user.name "Saurabh Barjatiya"
    git config --global user.email "saurabh@sbarjatiya.com"
    git config --global color.ui true
    git config --global color.status auto
    git config --global color.branch auto
    git config --global core.editor vim
  2. Configure to use simple push by default
    git config --global push.default simple
    Refer https://stackoverflow.com/questions/21839651/git-what-is-the-difference-between-push-default-matching-and-simple


Postfix and logwatch

  1. Configure postfix gmail relay using Sending email via Gmail relay through postfix
  2. Edit /etc/aliases and add saurabh [at] rekallsoftware.com as alias for root.
  3. newaliases
  4. Install logwatch
  5. systemctl start postfix
  6. systemctl enable postfix


Configure SSH

  1. Symbolic link ssh from /mnt/data1
    ln -s /mnt/data1/workspace/application-data/ssh .ssh
  2. ssh root@sbarjatiya.com
    Enter passphrase (4cs..) and enable automatic unlock during login


Configure thunderbird

  1. Create symbolic link
    ln -s /mnt/data1/workspace/application-data/thunderbird/ .thunderbird
  2. Run thunderbird and validate email accounts are accessible


Configure firefox

  1. Ensure that /home/saurabh/.mozilla is symlink to application-data folder
  2. Starting firefox and see that various profiles are appearing properly.
  3. Validate history and stored passwords are working as expected


Configure nautlius

  1. Open nautilus
  2. Create bookmarks for (Open folder and press Ctrl+D)
    • /mnt/data1/workspace/gbb/owncloud/saurabh
    • /mnt/data1
    • /mnt/data1/plain_folders/documents/Desktop/rekall
  3. Configure nautilus to open .org file via emacs by default, if not done already
  4. vim ~/.config/user-dirs.dirs #As saurabh user
    Comment all except Desktop and Downloads
  5. vim /etc/xdg/user-dirs.default #As root user
    Comment all except Desktop and Downloads
  6. Logoff and login again
  7. Open nautilus and remove bookmarks for Documents, Music, Videos and Pictures

Refer:


Configure /etc/hosts

  1. Various AWS backup VM IPs
  2. rekallcm1 to its IPv4 LAN IP


Configure owncloud client to sync /mnt/data1/workspace/gbb/owncloud while keeping local data

  1. Choose to sync folders up to 1000MB


Install flash-plugin

  1. Open https://get.adobe.com/flashplayer/
  2. Choose download of "Yum for Linux"
  3. Install downloaded rpm file
    yum -y localinstall adobe-release-x86_64-1.0-1.noarch.rpm
  4. Close all browsers (Esp. firefox)
  5. yum -y install flash-plugin
  6. Start firefox and check "about:plugins"


Install and configure fail2ban

CentOS 7.x fail2ban


Enable bind

  1. yum -y install bind bind-utils
  2. Configure bind for starting automatically:
    systemctl start named
    systemctl enable named
  3. Change DNS of machine in /etc/sysconfig/network-scripts/ifcfg-br0 file to 127.0.0.1
  4. Optionally edit /etc/named.conf as follows:
    • Update first listen to 'listen-on port 53 { 127.0.0.1; any; };'
    • Comment listen-on-v6 port 53 { ::1; };' using '//' as prefix
    • Update allow-query to 'allow-query { localhost; 10.0.0.0/8; 192.168.0.0/12; 172.16.0.0/16; };'
  5. Disable IPv6 lookups using CentOS 7.x Disabling IPv6 lookups in bind
  6. Restart named if not done already
  7. Test by running
    nslookup www.google.com 127.0.0.1


Make sure printer is recognized and working

  1. yum -y install hplip gutenprint-cups


Setup various servers and services

Configure software bridge

  1. Refer Creating bridge interfaces (br0) for virtual hosts to use shared interface section "Bridge interfaces in CentOS 7.0"
    • Include all IPv6, IPv4 and other addressing configuration in ifcfg-br0
  2. Validate that bridge is having ip address and not interface:
    nmcli connection show
    nmcli connection show <ethernet-interface> | grep br0
    Should show
    connection.master: br0
  3. Might have to reboot for things to work

Refer:


Re-add various older VMs

  1. Add the VMs via disk or config file
    1. Start virt-manager as normal non-root (eg saurabh) user. Add connection to Qemu/KVM if not present.
    2. Go to File -> New virtual machine
    3. Choose "Import existing disk image"
    4. Select VM disk (Eg =/mnt/data1/large_files/vms/tally_vm/tally_vm.qcow2= )
    5. Select OS type "Linux" and Version "CentOS 7.0"
    6. Give required RAM and CPU (Eg 2048, 2 vCPU)
    7. Ensure that bridge br0 is selected
    8. Give appropriate VM name (Eg tally_vm)
    9. Select "Customize configuration before install"
    10. For "Disk 1" select "Disk Bus" as "IDE" instead of virtio
    11. For CPU select copy host CPU configuration option
    12. For Windows XP on NIC choose "e1000" instead of default virtio
    13. Click "Begin installation" to boot the VM.
    14. Download Intel Pro/1000 MT drivers for 32-bit and 64-bit windows from: https://downloadcenter.intel.com/download/18717/Network-Adapter-Drivers-for-Windows-XP-Final-Release?product=50485 As such 32-bit drivers from above link works for Tally_VM. But for other OS / VMs consider using one of the below as required:
    15. Copy the downloaded setup to pen drive and make it available to VM using USB redirection.
    16. Install the setup and Ethernet should get detected.
  2. Take latest snapshot of VM and delete older snapshots via qemu-img
    1. Start virt-manager and take at least one snapshot of all important VMs.
    2. If there are VM folders in /mnt/data1/large_files/vms which are not useful to be added then delete those folders
    3. Go to vms folder and list snapshots of existing vm disks:
      cd /mnt/data1/large_files/vms/tally_vm
      qemu-img snapshot -l tally_vm.qcow2
      qemu-img snapshot -d '<Old-snapshot-name>' tally_vm.qcow2
      Be careful and do not delete recent snapshot taken
  3. Boot the VM and validate that it is still booting properly


Setup apache, mariadb including DB restore and php for wikis

  1. Replace httpd.conf from configuration folder:
    mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf-old
    ln -s /documents/room-documents/documents/general/configuration_files/httpd.conf /etc/httpd/conf/
  2. Install mariadb
    yum -y install mariadb-server
  3. Start and enable mariadb
    systemctl start mariadb
    systemctl enable mariadb
  4. Check old wiki passwords at following locations:
    • /documents/public_html/notes_wiki/LocalSettings.php
    • /documents/room-documents/documents/databases/mysql/notes_wiki/mediawiki/LocalSettings.php
    • /documents/room-documents/documents/databases/mysql/research_wiki/mediawiki/LocalSettings.php
    • /documents/room-documents/documents/databases/mysql/readme_wiki/mediawiki/LocalSettings.php
  5. Login as root user ('mysql -u root') and create other database accounts:
    create database wikidb_notes;
    grant all on wikidb_notes.* to wikidb_notes@localhost identified by '<password>';
    create database wikidb_res;
    grant all on wikidb_res.* to wikidb_res@localhost identified by '<password>';
    create database wikidb_readme;
    grant all on wikidb_readme.* to wikidb_readme@localhost identified by '<password>';
    create database notes_wiki;
    grant all on notes_wiki.* to notes_wiki@localhost identified by '<password>';
    flush privileges;
    \q
  6. Restore various database backups using:
    cd /documents/public_html/
    bunzip2 -k notes_wiki.sql.bz2
    cat notes_wiki.sql | mysql -u notes_wiki -p<password> notes_wiki
    rm -f notes_wiki.sql
    cd /documents/room-documents/documents/databases/mysql/notes_wiki/
    bunzip2 -k wikidb_notes.sql.bz2
    cat wikidb_notes.sql | mysql -u wikidb_notes -p<password> wikidb_notes
    rm -f wikidb_notes.sql
    cd /documents/room-documents/documents/databases/mysql/research_wiki
    bunzip2 -k wikidb_res.sql.bz2
    cat wikidb_res.sql | mysql -u wikidb_res -p<password> wikidb_res
    rm -f wikidb_res.sql
    cd /documents/room-documents/documents/databases/mysql/readme_wiki
    bunzip2 -k wikidb_readme.sql.bz2
    cat wikidb_readme.sql | mysql -u wikidb_readme -p<password> wikidb_readme
    rm -f wikidb_readme.sql
  7. Install required packages for php and php-mysql:
    yum -y install php-mysqlnd php php-xml httpd mod_ssl php-mbstring php-json
  8. Restart and enable httpd:
    systemctl restart httpd
    systemctl enable httpd


Setup postgreSQL databases

  1. Install postgreSQL and start DB
    yum -y install postgresql postgresql-server
    postgresql-setup --initdb --unit postgresql
    systemctl start postgresql
    systemctl enable postgresql
  2. Restore postgreSQL configuration file to allow password based logins:
    mv /var/lib/pgsql/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf-old
    sudo ln -s /documents/room-documents/documents/general/configuration_files/pg_hba.conf /var/lib/pgsql/data/
    sudo chown postgres:postgres /documents/room-documents/documents/general/configuration_files/pg_hba.conf
    systemctl restart postgresql
  3. Read existing DB password in
    • /documents/room-documents/public_html/bookworm/global/library/global_common.php
  4. Use below to create required databases:
    sudo su - postgres
    psql
    CREATE USER saurabh WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
    CREATE DATABASE saurabh WITH OWNER=saurabh;
    CREATE USER sen WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
    CREATE DATABASE sen WITH OWNER=sen;
    \q
    exit
  5. Restore various database backups using:
    cd /documents/room-documents/documents/databases/pgsql/
    psql -U saurabh -d saurabh < saurabh_saurabh.sql
    # cd /documents/room-documents/documents/databases/pgsql/
    # bunzip2 -k sen.sql.bz2
    # psql -U sen -d sen < sen.sql
    # rm -f sen.sql
    Note that localhost/bookworm does not seems to work with latest php packages
  6. Install php postgresql libraries:
    yum -y install php-pgsql
    systemctl restart httpd


Configure backups for localhost and AWS VMs

  1. Install required packages
    yum -y install rsnapshot sshpass
  2. Create rsnapshot root folder:
    mkdir -p /mnt/data1/large_files/rsnapshot_backups/
  3. Edit /etc/rsnapshot.conf as follows ***TABS and not spaces***
    snapshot_root /mnt/data1/large_files/rsnapshot_backups/
    no_create_root 1
    cmd_ssh /usr/bin/ssh
    retain hourly 6
    retain daily 7
    retain wekkly 5
    retain monthly 3
    logfile /var/log/rsnapshot
    link_dest 1
    backup /etc/ rekallcm1/
    backup /root/ rekallcm1/
    backup /var/log/ rekallcm1/
    backup /mnt/data1/erlang/ rekallcm1/
    backup /mnt/data1/personal/ rekallcm1/
    backup /mnt/data1/plain_folders/ rekallcm1/
    backup /mnt/data1/workspace/ rekallcm1/
    backup /mnt/data1/setup/ rekallcm1/
    backup /mnt/data1/windows_setup/ rekallcm1/
    backup_script /usr/bin/sshpass -p <password> mysqldump -u root -p wikidb_notes | bzip2 > /documents/room-documents/documents/databases/mysql/notes_wiki/wikidb_notes.sql.bz2 rekallcm1/.ignore1/
    backup_script /usr/bin/sshpass -p <password> mysqldump -u root -p wikidb_res | bzip2 > /documents/room-documents/documents/databases/mysql/research_wiki/wikidb_res.sql.bz2 rekallcm1/.ignore2/
    backup_script /usr/bin/sshpass -p <password> mysqldump -u root -p wikidb_readme | bzip2 > /documents/room-documents/documents/databases/mysql/readme_wiki/wikidb_readme.sql.bz2 rekallcm1/.ignore3/
    #Notes_wiki backup is taken by /documents/public_html/update.sh manually
    # sbarjatiya.com
    backup root@sbarjatiya.com:/etc/ sbarjatiya.com/
    backup root@sbarjatiya.com:/home/ sbarjatiya.com/
    backup root@sbarjatiya.com:/root/ sbarjatiya.com/
    # mail.rekallsoftware.com
    backup root@mail.rekallsoftware.com:/etc/ mail.rekallsoftware.com/
    backup root@mail.rekallsoftware.com:/home/ mail.rekallsoftware.com/
    backup root@mail.rekallsoftware.com:/root/ mail.rekallsoftware.com/
    backup root@mail.rekallsoftware.com:/var/www/html/ mail.rekallsoftware.com/
    # km.rekallsoftware.com
    backup root@km.rekallsoftware.com:/etc/ km.rekallsoftware.com/
    backup root@km.rekallsoftware.com:/home/ km.rekallsoftware.com/
    backup root@km.rekallsoftware.com:/root/ km.rekallsoftware.com/
  4. Create root ssh keys
    1. ssh-keygen
    2. Allow authorized access for root to
      • sbarjatiya.com
      • mail.rekallsoftware.com
      • km.rekallsoftware.com
    3. SSH as root to all the above servers and accept ssh fingerprint
  5. Test backups using
    rsnapshot hourly
  6. Edit root crontab using 'crontab -e' and use:
    0 0,4,8,12,16,20 * * * /usr/bin/rsnapshot hourly
    45 23 * * * /usr/bin/rsnapshot daily
    30 23 * * 0 /usr/bin/rsnapshot weekly
    15 23 1 * * /usr/bin/rsnapshot monthly


Take fresh full backup

Take full backup on external hdd

Take fresh full backup on external hdd using:

umount /dev/sdd1
mkdir -p /mnt/usb
mount /dev/sdd1 /mnt/usb

mkdir yyyy-mm-dd-rekallcm1-backup
cd yyyy-mm-dd-rekallcm1-backup

mkdir -p mnt/data1
rsync -aH --delete /etc/  etc/
rsync -aH --delete /home/  home/
rsync -aH --delete /root/ root/
rsync -aH --delete --exclude '/cache' /var/ var/
rsync -aH --delete --exclude '/large_files/rsnapshot_backups' /mnt/data1/  mnt/data1/

Refer:


<yambe:breadcrumb self="Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities">CentOS 8.x New machine configuration|New machine configuration</yambe:breadcrumb>