Difference between revisions of "CentOS 7.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities"
m |
m |
||
Line 267: | Line 267: | ||
===Install Google chrome=== | ===Install Google chrome=== | ||
# Refer [[CentOS 7.x | # Refer [[CentOS 7.x Installing Google chrome]] | ||
# Start chrome | # Start chrome | ||
# Do not make chrome default browser | # Do not make chrome default browser |
Revision as of 14:20, 27 October 2019
<yambe:breadcrumb>CentOS_7.x_New_machine_configuration|CentOS 7.x New machine configuration</yambe:breadcrumb>
CentOS 7.x Configure rekallcm1 (high-end desktop) with software raid, lxc, kvm and other utilities
Backup before formatting
Before formatting ensure that there is backup for on external drive
- /etc
- /home
- /root
- /mnt/data1 appropriate sub-folders
- Last hourly.0 backup of cloud VMs excluding local backups
For backup to external disk, use steps similar to:
mount /dev/sd<n>1 /mnt/usb mkdir /mnt/usb/2019-08-07-rekallcm1-backup rsync -axH /etc/ /mnt/usb/2019-08-07-rekallcm1-backup/etc/ rsync -axH /home/ /mnt/usb/2019-08-07-rekallcm1-backup/home/ cd /mnt/data1 mkdir -p /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1 rsync -axH plain_folders workspace /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1/ rsync -axH large_files --exclude=large_files/rsnapshot_backups /mnt/usb/2019-08-07-rekallcm1-backup/mnt/data1/ rsync -axH large_files/rsnapshot_backups/hourly.0 --exclude=hourly.0/rekallcm1 /mnt/usb/2019-08-07-rekallcm1-backup/rsnapshot_backups
Ensure that there is enough space in external disk (=df -h=) as required for various folders (=du -sh=). It is also important to check health of backup disk (=smartctl=) against failed self-test or other pre-failure alert signals.
OS installation
Install OS using following options (Assuming 3 disks of 2TB each):
- If necessary validate hardware information eg processor, disks, RAM, etc. You can look at =smartctl= information of disks for results of last self-tests and any possible pre-failure alerts.
- Partitioning:
- /boot/efi
- 1GB - EFI filesytem - RAID 1
- /
- 50GB - ext4 - RAID 1
- swap
- 18GB - swap - LVM with Volume group using RAID 5
- /mnt/data1
- Remaining - ext4 - LVM with Volume group using RAID 5
- Disable kdump
- Do not apply any security profile
- Server with GUI without Infiband, smart card, FTP related modules
- Enable networking and configure appropriate hostname (Eg rekallcm1)
- Create saurabh user with administrative privileges
Note that OS installation would be very slow while the RAID is building on large 3.8TB /mnt/data1 partitioning. It would take close to 5-6 hours for the installation to complete when installing through USB stick.
Initial configuration and package installations
Enable NOPASSWD based sudo access
- Login with saurabh user
- Enter saurabh user password after typing =sudo su -=
- Run 'visudo' and comment below line
- %wheel ALL=(ALL) ALL
- Uncomment below line
- # %wheel ALL=(ALL) NOPASSWD: ALL
- Exit root terminal and test by typing =sudo su -=. This time there should not be any password prompt.
IP addressing
- Configure static IP address using Network Manager GUI.
- Add the same IP in other hosts /etc/hosts files for easy access via name.
- Validate network connectivity
- ping 4.2.2.2
- Validate DNS resolution
- ping www.google.co.in
Note that this temporary. We will configure software bridge later for VMs and containers and disable access to interface via Network Manager (NM_CONTROLLED=no)
Configure history retention
Refer Storing date / time along with commands in history
After this exit all running shells and reopen new shell. Validate history is stored with date/time information.
Full update
- Edit /etc/yum.conf and set =keepcache=1=
- Do full update. This is required so that if any packages that were used before formatting were updated and have updated db/config files, those things can be read properly without version related issues:
- yum -y update --skip-broken
- Reboot into latest stable kernel
If the RAID partition rebuild is still not finished the full-update would run very slow, same as OS installation. Check software RAID rebuild status using
cat /proc/mdstat
Disable screen lock
Start gnome-control-center and under Privacy, disable automatic screen-lock
Disable SELinux
- setenforce 0
- edit /etc/sysconfig/selinux
- SELINUX=disabled
This is required as SELinux related errors are very hard to troubleshoot as the issues happen elsewhere apache, image editor, etc. and it is hard to connect that something is not working due to SELinux.
Restore backup to /mnt/data1
- Install byobu for running tasks in background
- yum -y install epel-release
- yum -y install byobu
- Mount external drive and copy files to /mnt/data1 after the raid sync operation has finished (Could take 4-5 hours for 3.8TB):
- mkdir /mnt/usb
- mount /dev/sd<n>1 /mnt/usb
- byobu
- #Inside byobu shell
- rsync -axH /mnt/usb/2019-08-06-rekallcm1-backup /mnt/data1/
- #Press F6 to exit, Run byobu again to check rsync has completed or not
- After backup is complete
- umount /mnt/usb
- Move backup folders to appropriate place Be careful with relative vs absolute path while doing below operations
- cd /mnt/data1/2019-08-06-rekallcm1-backup
- mv /mnt/data1/2019-08-06-rekallcm1-backup/mnt/data1/* /mnt/data1/
- rmdir -p mnt/data1
- #move rsnapshot_backups inside /mnt/data1/large_files with latest cloud VM backups inside hourly.0 folder
- mv home/* /home/*
- mv -f home/saurabh/* home/saurabh/.* /home/saurabh/
- ls -al home/saurabh #Ensure empty except local, gnupg, dbus, config, cache
- #For example if firefox was opened as saurabh user before restoring backup then
- #rm -rf /home/saurabh/.mozilla
- #mv home/saurabh/.mozilla
- #After home/saurabh is empty
- rm -rf home/saurabh
- rmdir -p home
- ls -al /home/saurabh #Ensure .emacs, .thunderbird, .mozilla are symlinks
- ls -l /home
- useradd jitender #UID 1001
- useradd achal #UID 1002
- passwd jitender
- passwd achal
- mv -f root/* root/.* /root/
- ln -s /mnt/data1/plain_folders/documents/ /documents
- #exit and ssh again to rekallcm1
- #Validate older history from backup is visible
- history | less
Install and configure required packages
Yum based packages
yum -y remove bind-chroot yum -y install htop #Colorful command-line top yum -y install gimp #Advanced image editor useful for editing pdf as images yum -y install kile #Latex editor yum -y install wireshark wireshark-gnome #Packet capturing yum -y install erlang yaws #Programming yum -y install fdupes #Duplicate file detection yum -y install dia #Diagram such as flowcharts or network design editor yum -y install libreoffice #Libreoffice yum -y install p7zip #Support for 7zip yum -y install kolourpaint #Basic image editor yum -y install expect #For shell automation. Includes mkpasswd utility yum -y install git bzr kdiff3 #For versioning and comparing files yum -y install bind bind-utils #For DNS related tasks yum -y install unetbootin liveusb-creator #For creating live bootable USB from iso files
Install and configure owncloud
- Install owncloud client using CentOS 7.x Owncloud desktop client installation
- ssh -X saurabh@rekallcm1
- owncloud &
- Local folder - /mnt/data1/workspace/gbb/owncloud
- Ensure that sync is happening properly
- Resolve all conflicts
- Might require deleting older renamed files and restarting owncloud client
- exit from ssh
Install and configure emacs
- yum -y install emacs
- Ensure /home/saurabh/.emacs and /home/saurabh/.emacs.d are symlink to appropriate folders of application-data
- Validate things are working by opening to-do.org file
- Configure nautilus to open .org file via emacs by default
Install vlc
Refer CentOS 7.x Installing VLC
Configure nautlius
From this point onwards we can work directly on rekallcm1, instead of working remotely
- Open nautilus
- Remove /home/saurabh/{Music,Pictures,Videos,Templates}
- Create bookmarks for
- /mnt/data1/workspace/gbb/owncloud/saurabh
- /mnt/data1
- /mnt/data1/plain_folders/documents/Desktop/rekall
- Configure nautilus to open .org file via emacs by default, if not done already
Configure firefox
- Ensure that /home/saurabh/.mozilla is symlink to application-data folder
- Starting firefox and see that various profiles are appearing properly.
- If required use following to restore .mozilla again from backup:
- mount /mnt/sd<n>1 /mnt/usb
- cd /mnt/usb/2019-08-06-rekallcm1-backup/mnt/data1/workspace/application-data/mozilla
- rsync -axH --delete ./ /mnt/data1/workspace/application-data/mozilla/
- umount /mnt/usb
- Validate history and stored passwords are working as expected
Install and configure thunderbird
- yum -y install thunderbird
- Launch thunderbird and validate that various emails accounts are visible
Install anydesk
- Refer CentOS 7.x installing anydesk
- As such below should work:
- yum -y localinstall /mnt/data1/large_files/setup/anydesk/anydesk-5.1.1-1.el7.x86_64.rpm
- Start anydesk and validate older saved IDs are coming properly
Configure SSH
- Validate /home/saurabh/.ssh folder has properly restored backup
- ssh root@sbarjatiya.com
- Enter passphrase and enable automatic unlock during login
Install Google chrome
- Refer CentOS 7.x Installing Google chrome
- Start chrome
- Do not make chrome default browser
- Disable submitting of usage information
- Test by opening www.google.com
Install flash-plugin
- Open https://get.adobe.com/flashplayer/
- Choose download of "Yum for Linux"
- As such below should work:
- yum -y localinstall /mnt/data1/large_files/setup/flash/adobe-release-x86_64-1.0-1.noarch.rpm
- Close all browsers (Esp. firefox)
- yum -y install flash-plugin
- Start firefox and check "about:plugins"
Configure git
- Configure various git parameters:
- git config --global user.name "Saurabh Barjatiya"
- git config --global user.email "saurabh@sbarjatiya.com"
- git config --global color.ui true
- git config --global color.status auto
- git config --global color.branch auto
- git config --global core.editor vim
- Configure to use simple push by default
- git config --global push.default simple
Configure bazaar
- Configure bazaar parameters:
- bzr whoami "Saurabh Barjatiya <saurabh@sbarjatiya.com>"
Enable bind
- Configure bind for starting automatically:
- systemctl start named
- systemctl enable named
- Change DNS of machine in /etc/sysconfig/network-scripts/ifcfg-br0 file to 127.0.0.1
- Optionally edit /etc/named.conf as follows:
- Update first listen to 'listen-on port 53 { 127.0.0.1; any; };'
- Comment listen-on-v6 port 53 { ::1; };' using '//' as prefix
- Update allow-query to 'allow-query { localhost; 10.0.0.0/8; 192.168.0.0/12; 172.16.0.0/16; };'
- CentOS 7.x Disabling IPv6 lookups in bind
- Restart named if not done already
Setup various servers and services
Configure software bridge
- Refer Creating bridge interfaces (br0) for virtual hosts to use shared interface section "Bridge interfaces in CentOS 7.0"
- Validate that bridge is having ip address and not interface:
- ip addr show
- brctl show
Install and configure lxc
- Install lxc by referring CentOS 7.x installing lxc
Import various /etc/hosts entries
- Open file /mnt/data1/2019-08-06-rekallcm1-backup/etc/hosts and copy valid entries and paste them in /etc/hosts file
Re-add various older VMs
Add the VMs via disk or config file
- Start virt-manager as root user. Add connection to Qemu/KVM if not present.
- virt-manager
- Go to File -> New virtual machine
- Choose "Import existing disk image"
- Select VM disk (Eg =/mnt/data1/large_files/vms/tally_vm/tally_vm.qcow2= )
- Select OS type "Linux" and Version "CentOS 7.0"
- Give required RAM and CPU (Eg 2048, 2 vCPU)
- Ensure that bridge br0 is selected
- Give appropriate VM name (Eg tally_vm)
- Select "Customize configuration before install"
- For "Disk 1" select "Disk Bus" as "IDE" instead of virtio
- For CPU select copy host CPU configuration option
- For Windows XP on NIC choose "e1000" instead of default virtio
- Click "Begin installation" to boot the VM.
- Download Intel Pro/1000 MT drivers for 32-bit and 64-bit windows from: https://downloadcenter.intel.com/download/18717/Network-Adapter-Drivers-for-Windows-XP-Final-Release?product=50485 As such 32-bit drivers from above link works for Tally_VM. But for other OS / VMs consider using one of the below as required:
- Download Linksys E1000 drivers from https://www.driverguide.com/driver/download/Linksys-E1000
- Download virtio drivers iso from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso
- Copy the downloaded setup to pen drive and make it available to VM using USB redirection.
- Install the setup and Ethernet should get detected.
Take latest snapshot of VM and delete older snapshots via qemu-img
- Start virt-manager and take at least one snapshot of all important VMs.
- If there are VM folders in /mnt/data1/large_files/vms which are not useful to be added then delete those folders
- Go to vms folder and list snapshots of existing vm disks:
- cd /mnt/data1/large_files/vms/tally_vm
- qemu-img snapshot -l tally_vm.qcow2
- qemu-img snapshot -d '<Old-snapshot-name>' tally_vm.qcow2
-
- Be careful and do not delete recent snapshot taken
- Boot the VM and validate that it is still booting properly
Postfix and logwatch
- Configure postfix gmail relay using Sending email via Gmail relay through postfix
- Edit /etc/aliases and add barjatiya.saurabh [at] gmail.com as alias for root.
- Install and configure logwatch
Setup apache, mariadb including DB restore and php for wikis
- Replace httpd.conf from configuration folder:
- mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf-old
- ln -s /documents/room-documents/documents/general/configuration_files/httpd.conf /etc/httpd/conf/
- Start and enable mariadb
- systemctl start mariadb
- systemctl enable mariadb
- Check old wiki passwords at following locations:
- /documents/public_html/notes_wiki/LocalSettings.php
- /documents/room-documents/documents/databases/mysql/notes_wiki/mediawiki/LocalSettings.php
- /documents/room-documents/documents/databases/mysql/research_wiki/mediawiki/LocalSettings.php
- /documents/room-documents/documents/databases/mysql/readme_wiki/mediawiki/LocalSettings.php
- Login as root user (=mysql -u root=) and create other database accounts:
- create database wikidb_notes;
- grant all on wikidb_notes.* to wikidb_notes@localhost identified by '<password>';
- create database wikidb_res;
- grant all on wikidb_res.* to wikidb_res@localhost identified by '<password>';
- create database wikidb_readme;
- grant all on wikidb_readme.* to wikidb_readme@localhost identified by '<password>';
- create database notes_wiki;
- grant all on notes_wiki.* to notes_wiki@localhost identified by '<password>';
- flush privileges;
- \q
- Restore various database backups using:
- cd /documents/public_html/
- bunzip2 -k notes_wiki.sql.bz2
- cat notes_wiki.sql | mysql -u notes_wiki -p<password> notes_wiki
- rm -f notes_wiki.sql
- cd /documents/room-documents/documents/databases/mysql/notes_wiki/
- bunzip2 -k wikidb_notes.sql.bz2
- cat wikidb_notes.sql | mysql -u wikidb_notes -p<password> wikidb_notes
- rm -f wikidb_notes.sql
- cd /documents/room-documents/documents/databases/mysql/research_wiki
- bunzip2 -k wikidb_res.sql.bz2
- cat wikidb_res.sql | mysql -u wikidb_res -p<password> wikidb_res
- rm -f wikidb_res.sql
- cd /documents/room-documents/documents/databases/mysql/readme_wiki
- bunzip2 -k wikidb_readme.sql.bz2
- cat wikidb_readme.sql | mysql -u wikidb_readme -p<password> wikidb_readme
- rm -f wikidb_readme.sql
- Install php7 for latest notes_wiki
- yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
- yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
- yum -y install yum-utils
- yum-config-manager --enable remi-php72
- yum -y update php
- Install required packages for php and php-mysql:
- yum -y install php-mysql php php-xml httpd mariadb-server mariadb mod_ssl php-mbstring
- Restart and enable httpd:
- systemctl restart httpd
- systemctl enable httpd
Setup postgreSQL databases
- Install postgreSQL and start DB
- yum -y install postgresql postgresql-server
- postgresql-setup initdb
- systemctl start postgresql
- systemctl enable postgresql
- Restore postgreSQL configuration file to allow password based logins:
- mv /var/lib/pgsql/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf-old
- sudo ln -s /documents/room-documents/documents/general/configuration_files/pg_hba.conf /var/lib/pgsql/data/
- sudo chown postgres:postgres /documents/room-documents/documents/general/configuration_files/pg_hba.conf
- systemctl restart postgresql
- Read existing DB password in
- /documents/room-documents/public_html/bookworm/global/library/global_common.php
- Use below to create required databases:
- sudo su - postgres
- psql
- CREATE USER saurabh WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
- CREATE DATABASE saurabh WITH OWNER=saurabh;
- CREATE USER sen WITH NOSUPERUSER NOCREATEDB LOGIN ENCRYPTED PASSWORD '<password>';
- CREATE DATABASE sen WITH OWNER=sen;
- \q
- exit
- Restore various database backups using:
- cd /documents/room-documents/documents/databases/pgsql/
- psql -U saurabh -d saurabh < saurabh_saurabh.sql
- # cd /documents/room-documents/documents/databases/pgsql/
- # bunzip2 -k sen.sql.bz2
- # psql -U sen -d sen < sen.sql
- # rm -f sen.sql
- Install php postgresql libraries:
- yum -y install php-pgsql
- systemctl restart httpd
-
- Note that localhost/bookworm does not seems to work with latest php packages
Configure backups for localhost and AWS VMs
- Install required packages
- yum -y install rsnapshot sshpass
- Copy rsnapshot backup file to /etc
- mv /etc/rsnapshot.conf /etc/rsnapshot.conf-old
- cd /mnt/data1/2019-08-06-rekallcm1-backup
- cp etc/rsnapshot.conf /etc/
- Create rsnapshot root folder:
- mkdir -p /mnt/data1/large_files/rsnapshot_backups/
- Edit /etc/rsnapshot.conf and comment backups which are not required or incorrect
- Test backups using
- rsnapshot hourly
- Edit root crontab using =crontab -e= and use:
- 0 0,4,8,12,16,20 * * * /usr/bin/rsnapshot hourly
- 45 23 * * * /usr/bin/rsnapshot daily
- 30 23 * * 0 /usr/bin/rsnapshot weekly
- 15 23 1 * * /usr/bin/rsnapshot monthly
Also see:
- Configuring laptop after Cent-OS re-installation
- Configure high-end desktop as server for lab experiments
<yambe:breadcrumb>CentOS_7.x_New_machine_configuration|CentOS 7.x New machine configuration</yambe:breadcrumb>