====== Debian Template VM ====== To speed-up later VMs deployments, you better start creating a VM template that will fit most of your later needs as a base OS. One of the most stable, versatile and full-featured distribution which is extremely well supported by an extended community throughout the world is **[[https://www.debian.org/|Debian]]**.\\ This is the reason we'll use this distro as our base OS to setup our fresh VM template. This template will be based on the latest [[https://www.debian.org/releases/stable/|Debian Stable release]] which, at the time of writing, is **Debian Stretch** (9.4.0). **PLEASE NOTE:** although all of the installation process steps will be identical whatever your virtualization environment, what follows includes steps specifically intended to prepare a VM that will run within a [[https://www.virtualbox.org/|VirtualBox]] environment, we'll also try to present the steps required to build this VM in a XenServer environment. In case you'd like to know how to install VirtualBox on a Debian (LMDE 2) workstation, you can refer to [[http://wiki.strategicz.com/lmde/doku.php?id=packages:virtualbox|this other wiki article]]. ---- ===== Configure a Fresh VM ===== ==== Under VirtualBox ==== === Memory and Storage === Open the VirtualBox desktop application and use the "New" button to start creating a fresh virtual machine, we'll use the following: * Name: Debian-9.4.0 * Type: Linux * Version: Debains (64-bit) Let's allow 512M of RAM and an 8GB hard disk, you can change these values to fit your needs. * Memory: **512 MB** * Hard disk: Create a virtual hard disk now * VDI * Dynamically allocated * **8.00GB** Create the VM. === Networking === In our general purpose template, we'd like to have, by default, two inner networks: * **NAT** network allowing access to the Internet (via DHCP) * **Host-only** network allowing management access on a fixed IP In this way, the guest OS will never interfere with the network in which the host machine evolves, as it will use NAT and DHCP to the host for Internet access. We'll also have a persistent, fixed, IP address which we can use to access this specific VM through the Host-only network. VirtualBox offers a simple and elegant way to achieve this configuration. In the VirtualBox desktop application, go to the menu //**File > Preferences**// (Ctrl-G) and select //**Network**// in the left list. In the //**NAT Networks**//, double click on the existing //**NatNetwork**// entry (or click on the **+** icon on the right to create a new entry). Configure it as follow: * Network Name: NatNetwork * Network CIDR: 10.0.2.0/24 * Network Options * Support DHCP The 10.0.2.0/24 might need to be changed in case your local network is already using those sub-network addresses... Then click the //**Host-only Networks**// tab and click on the **+** icon on the right to create a new network if it doesn't already exist: * Adapter //**vboxnet0**// * IPv4 Address: 172.20.20.1 * IPv4 Network Mask: 255.255.255.0 Leave DHCP server empty for this config, feel free to use any private IP addresses range: * 10.0.0.0 to 10.255.255.255 * 172.16.0.0 to 172.31.255.255 * 192.168.0.0 to 192.168.255.255 ===== Debian ISO ===== Now we need the debian .iso file to initiate the VM OS install. All installation images for Debian are available on the [[https://www.debian.org/distrib/|Get Debian]] page, we'll use the [[https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso|64-bit PC netinst iso]] image, downloading it somewhere on our host system, lets say our downloaded file location is: //**/home//Downloads/debian-9.4.0-amd64-netinst.iso**// Let's now associate this iso file as the startup disk for our VirtualBox VM:\\ In the VirtualBox desktop application interface, having your fresh VM selected in the left column, click on the //**Storage**// tab. You should see a panel exposing 2 controllers: * Controller: IDE * Empty * Controller: SATA * Debian-9.4.0.vdi Select the //**empty**// entry by clicking on the CD disk icon, then, on the right, under **attributes** **"Optical Drive:"**, select **IDE Primary Master**, then click the CD icon next to the po-up menu and, in the dialog that opens, select the downloaded Debian .iso file you got from the previous step: //**/home//Downloads/debian-9.4.0-amd64-netinst.iso**// **Start** the VirtualBox VM using the **Start** button on top of the VitualBox window. ===== Debian Installer ==== Let's start our VM to delve into the Debian installation process (click on the green arrow in the VirtualBox interface). A window opens, showing you the Debian installation interface. As a general rule you can use the up, down, left, right and tab and Enter keys to navigate the menus the Debian installer will present you. Select the first option: **Graphical Install** (enter) Use up / down arrows keyboard keys or your mouse to select what's appropriate to your needs: * Language * Location * Locales (United States - en_US.UTF-8 is recommended) * Keyboard Select the first Ethernet controller (enp0s3) that is associated with the NAT network. Select a hostname for the system: **deb9** (continue)\\ Leave domain name **empty** Set root password.\\ New user: System Admin\\ Username for user account: sysad\\ Password:\\ Partitioning method: * Manual * SCSI (0,0,0) (sda) 8.6 GB ATA VBOX HARDISK * yes * Select FREE SPACE (continue) * Create a new partition * Leave default size (all disk) * Primary * Set bootable flag to **on** (double click or select and space) * Done setting up the partition * Finish partitioning and write changes to disk You'll be asked if you want to proceed without a swap partition, go ahead and say "no", as we'll create a **swap file** later-on.\\ This choice is aiming at simplifying potential subsequent needs of extending the hard disk partition, since we won't have to worry about partitioning scheme and will only have one single partition to extend. Write the changes to disk? ... yes Scan another DVD? ... No Configure the package manager: get back to the top of the list: enter information manually: * //**deb.debian.org**// as mirror hostname * directory: /debian/ * proxy: (leave blank) The //**http://deb.debian.org**// address is intended to always link your system to the most effective mirror related to the place where you're currently standing. Please refer to the [[http://deb.debian.org**//|deb.debian.org]] site for more information. Popularity contest ... pick your choice Software selection: * uncheck: Debian desktop environment * check only: ssh server + standard system utilities Accept the default GRUB loader master boot record location (/dev/sda). The Debian installation process has completed ! ===== Network Configuration ===== ==== NIC's ==== The first thing we'll do is setup the NICs configuration so that we'll have access to the system through an ssh connection. As we've linked 2 NICs to our VM, we'll use the one on the NAT network with DHCP, while configuring a static IP on the other, linked to the vboxnet0 virtual network. To do so, we use the console that VirtualBox presents for the running VM and log in as root. First we check the IP configuration: > ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 08:00:27:bc:55:8e brd ff:ff:ff:ff:ff:ff inet 10.0.2.15/24 brd 10.0.2.255 scope global enp0s3 valid_lft forever preferred_lft forever inet6 fe80::a00:27ff:febc:558e/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 08:00:27:44:2f:bd brd ff:ff:ff:ff:ff:ff We see that the NAT NIC is named ''enp0s3'' and has received an IP address, while the vboxnet0 NIC is referred to as ''enp0s8'', but this one is //DOWN//.\\ Let's change this by editing the ''/etc/network/interfaces'' configuration file. In order to assign a static IP on vboxnet0 for our VM and keep the DHCP on the NAT, something like the following should do: > nano /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The NAT network interface allow-hotplug enp0s3 iface enp0s3 inet dhcp # The vboxnet0 network interface allow-hotplug enp0s8 iface enp0s8 inet static address 172.20.20.10/24 Use CTRL+x, Y, Enter to save the file Restarting the network is sometimes tricky, so as long as we're still configurating our VM, the easiest way is to reboot, but don't be too fast ! ==== Login using an rsa key ==== For ease of use we'll want to be able to login as root using an rsa key, which will be uploaded using the ''ssh-copy-id'' command, but this requires us to be able to log in at least once using a password Let's prepare that by editing our ssh daemon configuration file ''/etc/ssh/sshd_config'': > nano /etc/ssh/sshd_config REPLACE (line 32) #PermitRootLogin prohibit-password WITH PermitRootLogin yes CTRL+x, y, Enter, to save the file. We're now ready to reboot: > reboot Once the VM has rebooted, we should be able to access it using a terminal on the host machine.\\ To make sure this works, let's ping our guest VM: > ping 172.20.20.10 PING 172.20.20.10 (172.20.20.10) 56(84) bytes of data. 64 bytes from 172.20.20.10: icmp_seq=1 ttl=64 time=0.511 ms 64 bytes from 172.20.20.10: icmp_seq=2 ttl=64 time=0.560 ms 64 bytes from 172.20.20.10: icmp_seq=3 ttl=64 time=0.547 ms ... Since the VM can be reached, let's upload our rsa key to it.\\ In case you don't have an rsa key pair on your computer, and don't know how to generate it, you can refer to this [[https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2|DigitalOcean Tutorial]] to learn how to do this. > ssh-copy-id -i ~/.ssh/id_rsa.pub root@172.20.20.10 /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@172.20.20.10's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@172.20.20.10'" and check to make sure that only the key(s) you wanted were added. Now that our key is on the guest VM, we'll log into it without having to enter a password. We'll immediately reinforce no password login for root once we're hooked: > nano /etc/ssh/sshd_config REPLACE (line 32) PermitRootLogin yes WITH PermitRootLogin prohibit-password CTRL+x, y, Enter, to save the file Note that this time the line is **uncommented**, this means we need an rsa key to login as root. All that is left is to reload our ssh daemon configuration to enforce the new configuration: > systemctl reload sshd ===== Swap File ===== As stated above, using a swap partition isn't always the best option when using a VM since it might become necessary to resize the swap space when the VM's memory configuration is changed based on its utilization. When a swap partition is used, it becomes necessary to "play" with the partitions sizes, which means changing the partitions scheme and oftentimes even the virtual disk's size, which is tedious... Therefore, in the case of VMs, it is often much easier to create the swap space using a system file, you can learn more about this in [[https://www.digitalocean.com/community/tutorials/how-to-configure-virtual-memory-swap-file-on-a-vps|this Digital Ocean's article]]. Here are the steps to create and use a swap file. First to make sure swap is not already activated on the system use the ''free'' command: > free total used free shared buff/cache available Mem: 2058304 37680 1759884 2968 260740 1872108 Swap: 0 0 0 The ''free : 0'' value on the ''Swap:'' line means swap is indeed **not activated** We'll create the partition file under ''/var'' and name it ''swap.img'', then change its permissions to ''600'' so no user will be able to access it: > touch /var/swap.img > chmod 600 /var/swap.img === Sizing === Deciding on the swap size is really case dependent. In general, it is recommended recommend to set it to 1-2x the available system RAM. So, if you have a 512mb RAM VM, use 512mb-1gb swap. If you have a 1gb RAM VM use 1gb-2gb swap, etc. This is not a hard and fast rule, for example if you have a 4gb RAM VM it may be best to use little (512mb) or no swap at all.\\ We use the ''dd'' command to stretch our swap file size, filling it with zeroes to the size we need (here 2Gb): > dd if=/dev/zero of=/var/swap.img bs=1024k count=2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.57258 s, 1.4 GB/s Next, we prepare the file to be usable as the swap file: > mkswap /var/swap.img Setting up swapspace version 1, size = 2 GiB (2147479552 bytes) no label, UUID=f8d9bdfe-4090-4bc6-8f7a-fc74b64946ad Then we turn on swapping: > swapon /var/swap.img > free total used free shared buff/cache available Mem: 2058304 38784 67972 2968 1951548 1847628 Swap: 2097148 0 2097148 We now see that the swap is active. We could turn off swapping with the ''swapoff /var/swap.img'' command.\\ Turning the swap on this way will **not activate swapping on the next boot**, so we need to modify ''/etc/fstab'' to have the swap file activated at boot time: > nano /etc/fstab ADD THOSE LINES # swap file /var/swap.img none swap sw 0 0 Finally, one could define the system's //swappiness//, which tells the Linux kernel/VM handler how likely it should be to use VM. It is a percent value, between 0 & 100. A usual recommendation for VMs is 30: > sysctl -w vm.swappiness=30 vm.swappiness = 30 To make swappiness persistent: > nano /etc/sysctl.conf ... ################################################################### # Swap usage # # vm.swappiness = 0 The kernel will swap only to avoid an out of memory condition # vm.swappiness = 1 Minimum amount of swapping without disabling it entirely. # vm.swappiness = 10 This value is sometimes recommended to improve performance when sufficient memory exists in a system. # vm.swappiness = 60 The default value. # vm.swappiness = 100 The kernel will swap aggressively. ################################################################### vm.swappiness = 10 That's about it !\\ If you want to know what your current system's swappiness is, use ''sysctl -a | grep swappiness''. ===== Shell Customization & Utilities ===== The default shell is sometimes a bit difficult to read, an there are a few useful utilities that we'd want to have at hand to manage our VM, so let's install our favorite ones. Note that the ''apt-get update'' and ''apt-get upgrade'' instructions are not necessary on a fresh install, they are here only to remind one that it's always best to update the system before a complementary package installation: ==== Additional Packages ==== > apt-get update > apt-get upgrade > apt-get install htop nload sysstat tmux ranger lnav We won't detail here and now what each package does or how to use it, but feel free to discover what each of those can help you do in the shell, you won't be disappointed... === htop === Could be considered as ''top'' on steroïds: [[https://www.maketecheasier.com/power-user-guide-htop/|The Power-User’s Guide to htop]] === nload === Will give you a graphic vision of the machine's NICs usage: [[https://www.tecmint.com/nload-monitor-linux-network-traffic-bandwidth-usage/|read more here]]. === sysstat === [[http://www.thegeekstuff.com/2011/03/sar-examples/|This article]] explains how to install and configure ''sysstat'' package (which contains sar utility), [[https://www.maketecheasier.com/monitor-linux-performance-with-sysstat/|This article]] can also give you some ideas on how to use ''sysstat'' to monitor the following Linux performance statistics using sar. * Collective CPU usage * Individual CPU statistics * Memory used and available * Swap space used and available * Overall I/O activities of the system * Individual device I/O activities * Context switch statistics * Run queue and load average data * Network statistics * Report sar data from a specific time === tmux === At the very basic ''tmux'' would be called a //terminal multiplexer//. Simply speaking, tmux acts as a window manager within your terminal and allows you to create multiple windows and panes within a single terminal window.\\ You are invited to consult this [[https://www.hamvocke.com/blog/a-quick-and-easy-guide-to-tmux/|Quick and Easy Guide to tmux]] as a starting point. Also this [[https://tmuxcheatsheet.com/|cheat sheet]] can be helpful once you start harnessing ''tmux'' powers. === ranger === ''ranger'' is a file manager that works in a terminal window and provides a smooth way to view and edit files. Here is a nice reference for ranger functions: [[http://ranger.nongnu.org/ranger.1.html]] === lnav === As it calls itself, ''lnav'' is //The Log File Navigator//. It's incredibly helpful when it comes to visualize and analyze all types of log files. You can start by reading [[https://lnav.readthedocs.io/en/latest/|the ''lnav'' documentation]] to get an idea of what it is capable of. ==== Bash Shell Configuration ==== ==== Colors ==== One of the main difficulty in reading the console is that, unconfigured, it is totally monochromatic.\\ Adding colors can help the user's eye catch the structure and meanings of the output. That's what we'll do now. === Shell Prompt === Spotting the shell prompt is important as it not only can bring some important info, like the user/server names and working directory, it also allows to visually separate each command output. Here is a way to greatly enhance your shell prompt, not only colorizing it, but choosing what info is displayed. The following example was created using the great tool available at [[http://bashrcgenerator.com/|http://bashrcgenerator.com/]], it's warmly recommended that you build your own prompt using this fantastic tool. # Colored prompt tty -s && export PS1="\[\033[38;5;130m\][\d]\[$(tput sgr0)\]\[\033[38;5;15m\] \[$(tput sgr0)\]\[\033[38;5;208m\]\t\[$(tput sgr0)\]\[\033[38;5;15m\]\n\[$(tput sgr0)\]\[\033[38;5;75m\]\w\[$(tput sgr0)\]\[\033[38;5;15m\]\n\[$(tput sgr0)\]\[\033[38;5;82m\]\u\[$(tput sgr0)\]\[\033[38;5;64m\]@\[$(tput sgr0)\]\[\033[38;5;71m\]\H\[$(tput sgr0)\]\[\033[38;5;15m\] > \[$(tput sgr0)\]" Note that, in our ''.bashrc'' code, we add ''tty -s &&'' in front of this to avoid errors when connecting through ssh from a non-tty instance (like rsync or zfs send/receive). **Colored prompt in tmux** As we plan to use tmux (a lot), we'll need to have it run in 256 colors, which is not the case by default. To fix this we need to alias the tmux command as well as modify the tmux configuration file: # tmux 256 colors for prompt compatibility alias tmux='TERM=xterm-256color tmux' set -g default-terminal "screen-256color" === Listing colors === Another enhancement that a colored shell can bring is an easy way to identify listed elements by their color. The simplest way to achieve this is by adding the following code in you ''~/.bashrc'' file: # Listing colors export LS_OPTIONS='--color=auto' eval "$(dircolors -b)" alias ls='ls $LS_OPTIONS' This will colorize your listing outputs with the preconfigured ''dircolors'' values. You could spend some time building your own preferred file types colors by using the references presented on [[http://linux-sxs.org/housekeeping/lscolors.html|this linux-sxs.org page]]. That would be used in your ''~/.bashrc'' file as follow: alias ls='ls --color' LS_COLORS='di=92:fi=0:ln=100:pi=5:so=5:bd=5:cd=5:or=31:mi=0:ex=95:*.rpm=90' export LS_COLORS === Aliases === Finally, aliases are a great way to simplify some commands usage. Here is one example that makes ''nano'' always display the line/character number you're editing: # CUSTOM ALIASES alias nano="nano -c" ===== System Configuration ===== ==== Pre-Login Message ==== As it might be convenient to have some information when starting up the VM, we'll modify the pre-login message as follow: > sudo nano /etc/issue Debian GNU/Linux 9 [\v] Host: \n TTY: \l Time: \d \t IP[enp0s3]: \4{enp0s3} IP[enp0s8]: \4{enp0s8} On next boot, the VM's initial login screen will display some info, notably IP addresses of ''eth0'' & ''eth1'' interfaces. ==== sudo ==== In case you want to allow users to up their privileges, you could install sudo to let them act as superusers: > su root password: > apt-get install sudo > usermod -a sysadmin -G sudo You might need to uncomment the sudo group members line in ''/etc/sudoers'' using **visudo**: [> su root] [password:] > visudo Check for line: # Allow members of group sudo to execute any command %sudo ALL=(ALL:ALL) ALL Make sure this line is **NOT** commented out. Please note that **you might have to re-connect** to a new session to have the user's sudo rights activated. ==== Time Sync ==== One important aspect when setting up a new system is time synchronization, to achieve this we'll use ''ntp''. Debian 9 should have time synchronization built in and activated by default using the standard ''ntpd'' time server, provided by the ''ntp'' package. First lets check whether the ''ntp'' time server is running: > service ntp status OR > systemctl status ntp If you get an answer of type : ''Unit ntp.service could not be found.'', you need to install, and maybe configure, the ''ntp'' and ''ntpdate'' packages, then check the ntp service has been started: > apt-get install ntp ntpdate > systemctl status ntp ● ntp.service - LSB: Start NTP daemon Loaded: loaded (/etc/init.d/ntp; generated; vendor preset: enabled) Active: active (running) since Thu 2018-11-22 16:38:06 UTC; 16s ago Docs: man:systemd-sysv-generator(8) CGroup: /system.slice/ntp.service └─27000 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 108:112 Nov 22 16:38:10 cloud ntpd[27000]: Soliciting pool server 37.187.104.44 Last, we want to setup the ''ntp'' service to automatically start at startup: > systemctl enable ntp You can refer to [[https://linuxconfig.org/how-to-setup-ntp-server-and-client-on-debian-9-stretch-linux|this page]] to learn more about the service's configuration. You can refer to [[https://blog.sleeplessbeastie.eu/2015/04/27/how-to-manage-system-services-on-debian-jessie/|this list of systemctl commands]] to see how to manage services. To configure your system's time zone use the ''dpkg-reconfigure tzdata'' ==== SMART ==== The following **only applies to physical servers** ''smartmontools'' is a set of applications that can test hard drives, automatically notify you when the failure rate rises and read the harddisk SMART statistics to detect failures early. For a complete //How To// check [[https://www.howtoforge.com/tutorial/monitor-harddisk-with-smartmon-on-ubuntu/|this article @howtoforge.com]]. The smartmontools package contains two utility programs (smartctl and smartd) to control and monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (SMART). > apt-get install smartmontools > lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 1.8T 0 disk ├─sda1 8:1 0 511M 0 part sdb 8:16 0 1.8T 0 disk ├─sdb1 8:17 0 511M 0 part sdc 8:32 1 1.8T 0 disk ├─sdc1 8:33 1 511M 0 part > smartctl -i /dev/sda smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.138-xxxx-std-ipv6-64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: HGST HUS726020ALA610 Serial Number: N4G3M6DY ... SMART support is: Available - device has SMART capability. SMART support is: Enabled Check the last 2 lines to make sure SMART is enabled on each device you want to monitor, if you need to enable it (it won't do any harm if already enabled): > smartctl -s on /dev/sda To check the complete SMART information for an IDE drive and SATA drive type: # (for IDE drives) > sudo smartctl -a /dev/sda # (for SATA drives) > sudo smartctl -a -d ata /dev/sda You can check your hard drive's status by typing: > sudo smartctl -H /dev/sda The tool used to test your hard drive has three types of tests to perform, Short, Extended & Conveyance.\\ You can check which is available by typing: > sudo smartctl -c /dev/sda ... Self-test supported. No Conveyance Self-test supported. ... Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 288) minutes. ... To effectively launch a test: > sudo smartctl -t short /dev/sda > sudo smartctl -t long /dev/sda > sudo smartctl -t conveyance /dev/sda === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Sending command: "Execute SMART Short self-test routine immediately in off-line mode". Drive command "Execute SMART Short self-test routine immediately in off-line mode" successful. Testing has begun. Please wait 2 minutes for test to complete. Test will complete after Sat Nov 24 00:12:47 2018 Use smartctl -X to abort test. To get the test results you will have to type: > sudo smartctl -l selftest /dev/sda === Sending Mail === It would be nice to have ''smartd'' automatically sending us emails whenever SMART detects something worth notifying.\\ This could be achieved by installing and configuring a MTA like a ''postfix'' server. Although, as we don't need (want) this server to become an smtp relay, we'll rather install the ''heirloom-mailx / s-nail'' package and configure it to use an external smtp relay, see the [[https://www.systutorials.com/1411/sending-email-from-mailx-command-in-linux-using-gmails-smtp/|this article @systurials.com]]. > apt-get install heirloom-mailx > cd /usr/bin/ > ln -s heirloom-mailx mailx We need to create the ''mailx'' symbolic link in order to have it available as such under the command line. As is stated in the [[https://linux.die.net/man/1/mailx|mailx man page]]: Normally, mailx invokes ''sendmail(8)'' directly to transfer messages. If the ''smtp'' variable is set, a SMTP connection to the server specified by the value of this variable is used instead. If the SMTP server does not use the standard port, a value of ''server:port'' can be given, with port as a name or as a number. Let's test an external smtp server using an all-in-one command: > echo "My message body" | mailx -v \ -s "Server Test Mail" \ -S smtp-use-starttls \ -S ssl-verify=ignore \ -S smtp-auth=login \ -S smtp=smtp.gmail.com:587 \ -S from="*******@gmail.com(System Watch)" \ -S smtp-auth-user=*******@gmail.com \ -S smtp-auth-password=******* \ recipient@domain.com Resolving host "smtp.gmail.com:587" ... done Connecting to "2a00:1450:400c:c06::6c:587" ...connected. You'll probably need to allow //Less secured apps access// in you gmail account for this to work ([[https://myaccount.google.com/lesssecureapps|here]]) Once this works, and you effectively receive the email at the ''recipient@domain.com'' address, it's time to configure a system wide smtp configuration for ''mailx'' and send a last test.\\ Note that, although the usage of the user specific ''~/.mailrc'' config file was quite obvious, it was much more tricky to determine the correct location (and name) of the global configuration file used by the ''mailx'' command. Having seen a lot of references to ''/etc/mail.rc'', it took a peak at the source code of ''s-nail'' to finally establish that, for v14.8.16, the correct location for the general configuration file was ''/etc/s-nail.rc''! > nano /etc/s-nail.rc TYPE account gmail { set smtp-use-starttls set ssl-verify=ignore set smtp=smtp.gmail.com:587 set smtp-auth=login set smtp-auth-user=*******@gmail.com set smtp-auth-password=******* set from="*******@gmail.com(System Watch)" } > echo "Global mailx configuration file was used here." | mailx -v -A gmail -s "Sent with global configuration" recipient@domain.com Resolving host "smtp.gmail.com:587" ... done Connecting to "2a00:1450:400c:c0b::6d:587" ...connected. When this works, we're ready to automate SMART reports... === Automating SMART Reports === First we'll need to run ''smartmontools'' as a system Deamon in order to have SMART tests run automatically, to do this we'll edit the ''smartmontools'' config file: > sudo nano /etc/default/smartmontools ... # uncomment to start smartd on system startup start_smartd=yes ... Now, ''smartd'' will be launched at boot time. Next, to define how SMART will scan the disk and what actions are to be taken if SMART returns any error, we'll configure ''smartd''.\\ By default, ''smartd'' will run ''/usr/share/smartmontools/smartd-runner'', which will create a temporary report file, and in turn, will run scripts located in ''/etc/smartmontools/run.d/''. As we'd like to modify this behavior, well create our own copy of those scripts: > mkdir -p ~/.smartd/run.d > cp /usr/share/smartmontools/smartd-runner ~/.smartd/ > cp /etc/smartmontools/run.d/10mail ~/.smartd/run.d/10mailx > nano ~/.smartd/smartd-runner #!/bin/bash -e run-parts --report --lsbsysinit --arg="$1" \ --arg="$2" --arg="$3" -- /root/.smartd/run.d > nano ~/.smartd/run.d/10mailx #!/bin/bash -e # Send mail if /usr/bin/mailx exists if ! [ -x /usr/bin/mailx ]; then echo "Your system does not have /usr/bin/mailx. Install the mailx package" exit 1 fi echo "$SMARTD_FULLMESSAGE" | /usr/bin/mailx -A gmail -s "$SMARTD_FAILTYPE - $SMARTD_MESSAGE" $SMARTD_ADDRESS These two scripts should now work together and generate a SMART report that gets emailed to a defined email address. In order to achieve this, we still need to configure the ''smartd'' service through the ''/etc/smartd.conf file''. Open it and uncomment the first line starting with ''DEVICESCAN'', replacing it as follow: > sudo nano /etc/smartd.conf ... DEVICESCAN -M test -a -H -l error -l selftest -f -s (S/../..././02|L/../../1/00) -m tech@tacticz.com -M exec /root/.smartd/smartd-runner ... OPTIONS:\\ * -M test : specifies that a test run should be executed the next time the ''smartd'' service is restarted * -a : This enables some common options. You almost certainly want to use it as it checks the SMART health status (-H). Reports increases in both SMART error logs (-l). To check for failure of any Usage Attributes (-f) . * -s (S/../.././02|L/../../1/00) : This schedules the short and long self-tests. In this example, the short self-test will run daily at 2:00 A.M. The long test will run on every Monday at 0:00. For more information, see [[https://www.freebsd.org/cgi/man.cgi?query=smartd.conf&manpath=ports&sektion=5|the smartd.conf man page]]. To test that everything works as expected, specifically that emails get sent, and since we have set the ''-M test'' option, we'll restart ''smartd''. As we restart the service, it is a good idea to have an eye on the log files so to get a feedback of the operations: > lnav /var/log/ > systemctl restart smartd You should see something like this in the logs: ... Nov 25 03:06:09 cloud smartd[433]: Opened configuration file /etc/smartd.conf ... Nov 25 03:06:09 cloud smartd[433]: Device: /dev/sda [SAT], state read from /var/lib/smartmontools/smartd.HGST_HUS726020ALA610-N4G3M6DY.ata.state ... Nov 25 03:06:09 cloud smartd[433]: Monitoring 3 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices Nov 25 03:06:09 cloud smartd[433]: Executing test of /root/.smartd/smartd-runner to recipient@domain.com Nov 25 03:06:09 cloud smartd[433]: Test of /root/.smartd/smartd-runner to recipient@domain.com: successful ... Test emails should be delivered to the target mailbox.\\ When it works, remove the ''-M test'' option in ''/etc/smartd.conf'' and restart the deamon with ''systemctl restart smartd''. === FAIL2BAN === On a physical server it might be a good idea to install ''fail2ban'' to establish a minimal protection. [[https://upcloud.com/community/tutorials/install-fail2ban-debian/]] ===== Install VirtualBox guest additions ===== **On Debian 8** > apt-get install build-essential module-assistant > m-a prepare **On Debian 9** Note that VirtualBox 5.2.12 was required to successfully complete those operations on a Debian 9.4.0 install. To get a complete explanation on how to install the latest VirtualBox on your host system please refer to [[https://www.virtualbox.org/wiki/Linux_Downloads|this virtualbox.org page]]. > sudo apt install -y linux-headers-amd64 > sudo apt build-dep -y linux From VM's window menu: Devices > Insert Guest Additions CD image... > mount /dev/cdrom /media/cdrom mount: /dev/sr0 is write-protected, mounting read-only > sh /media/cdrom/VBoxLinuxAdditions.run You'll need to reboot to have the guest additions loaded > reboot In case your new install requires the installation of the VirtualBox Extension Pack (to support USB2/3, NVMe etc...), download the latest Extension Pack as follow: > cd ~/Downloads > LatestVirtualBoxVersion=$(wget -qO - http://download.virtualbox.org/virtualbox/LATEST.TXT) && wget "http://download.virtualbox.org/virtualbox/${LatestVirtualBoxVersion}/Oracle_VM_VirtualBox_Extension_Pack-${LatestVirtualBoxVersion}.vbox-extpack" Then, in the VirtualBox UI, go to ''File > Preferences... > Extensions'' and use the green ''+'' icon to locate the previously downloaded vbox-extpack file. ===== Install XenServer Tools ===== In case you are working with XenServer, you first need to connect the ''xs-tools.iso'' to your newly installed VM. This can be achieved using a GUI application like [[https://xenserver.org/partners/developing-products-for-xenserver/21-xencenter-development/88-xc-dev-home.html|XenCenter]] (Windows only) or [[https://github.com/OpenXenManager/openxenmanager|OpenXenManager]] (Python > All platforms). You might also choose to install [[https://xen-orchestra.com/|Xen Orchestra]] as a VM straight on your XenServer platform, which will give you access to a nice Web GUI to manage your XenServer pools. If you prefer to use the command line interface, refer to [[http://wysotsky.info/%D1%81%D0%BE%D0%B7%D0%B4%D0%B0%D0%BD%D0%B8%D0%B5-%D0%B2%D0%B8%D1%80%D1%82%D1%83%D0%B0%D0%BB%D1%8C%D0%BD%D0%BE%D0%B9-%D0%BC%D0%B0%D1%88%D0%B8%D0%BD%D1%8B-%D0%B2-xenserver-%D0%B8%D0%B7-%D0%BA%D0%BE/|this (russian) article]] for the details. The command to link an ISO to a VM is something like: ''xe vm-cd-insert cd-name="name-label of device" vm="name-label of VM"''. Once the ''xs-tools.iso'' is connected to your debian VM, do the following **on the VM CLI** to install XenTools: > mount /dev/cdrom /mnt > bash /mnt/Linux/install.sh > umount /dev/cdrom In case you get a message like: mount: wrong fs type, bad option, bad superblock on /dev/sr0, missing codepage or helper program, or other error Try to issue the ''mount /dev/cdrom /mnt'' command a second time, this seems to work ! === Other OSs === To figure out how to install the XenServer Tools on other OSs, please refer to [[https://xen-orchestra.com/blog/install-xenserver-tools-in-your-vm/|this XenOrchestra web page]].