Differences

This shows you the differences between two versions of the page.

--- vms:debian [2018/11/24 03:25]
admin [SMART]
+++ vms:debian [2019/04/09 12:25] (current)
admin [Swap File]
@@ Line 269: / Line 269: @@
 > systemctl reload sshd
 </Code>
+===== Swap File =====
+As stated above, using a swap partition isn't always the best option when using a VM since it might become necessary to resize the swap space when the VM's memory configuration is changed based on its utilization. When a swap partition is used, it becomes necessary to "play" with the partitions sizes, which means changing the partitions scheme and oftentimes even the virtual disk's size, which is tedious...
+Therefore, in the case of VMs, it is often much easier to create the swap space using a system file, you can learn more about this in [[https://www.digitalocean.com/community/tutorials/how-to-configure-virtual-memory-swap-file-on-a-vps|this Digital Ocean's article]]. Here are the steps to create and use a swap file.
+First to make sure swap is not already activated on the system use the ''free'' command:
+<Code:bash>
+> free
+              total        used        free      shared  buff/cache   available
+Mem:        2058304       37680     1759884        2968      260740     1872108
+Swap:             0           0           0
+</Code>
+The ''free : 0'' value on the ''Swap:'' line means swap is indeed **not activated**
+We'll create the partition file under ''/var'' and name it ''swap.img'', then change its permissions to ''600'' so no user will be able to access it:
+<Code:bash>
+> touch /var/swap.img
+> chmod 600 /var/swap.img
+</Code>
+=== Sizing ===
+Deciding on the swap size is really case dependent. In general, it is recommended recommend to set it to 1-2x the available system RAM. So, if you have a 512mb RAM VM, use 512mb-1gb swap. If you have a 1gb RAM VM use 1gb-2gb swap, etc. This is not a hard and fast rule, for example if you have a 4gb RAM VM it may be best to use little (512mb) or no swap at all.\\
+We use the ''dd'' command to stretch our swap file size, filling it with zeroes to the size we need (here 2Gb):
+<Code:bash>
+> dd if=/dev/zero of=/var/swap.img bs=1024k count=2048
++0 records in
++0 records out
+2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.57258 s, 1.4 GB/s
+</Code>
+Next, we prepare the file to be usable as the swap file:
+<Code:bash>
+> mkswap /var/swap.img
+Setting up swapspace version 1, size = 2 GiB (2147479552 bytes)
+no label, UUID=f8d9bdfe-4090-4bc6-8f7a-fc74b64946ad
+</Code>
+Then we turn on swapping:
+<Code:bash>
+> swapon /var/swap.img
+> free
+              total        used        free      shared  buff/cache   available
+Mem:        2058304       38784       67972        2968     1951548     1847628
+Swap:       2097148           0     2097148
+</Code>
+We now see that the swap is active. We could turn off swapping with the ''swapoff /var/swap.img'' command.\\
+Turning the swap on this way will **not activate swapping on the next boot**, so we need to modify ''/etc/fstab'' to have the swap file activated at boot time:
+<Code>
+> nano /etc/fstab
+ADD THOSE LINES
+# swap file
+/var/swap.img    none    swap    sw    0    0
+</Code>
+Finally, one could define the system's //swappiness//, which tells the Linux kernel/VM handler how likely it should be to use VM. It is a percent value, between 0 & 100. A usual recommendation for VMs is 30:
+<Code:bash>
+> sysctl -w vm.swappiness=30
+vm.swappiness = 30
+</Code>
+To make swappiness persistent:
+<Code:bash>
+> nano /etc/sysctl.conf
+...
+###################################################################
+# Swap usage
+#
+# vm.swappiness = 0         The kernel will swap only to avoid an out of memory condition
+# vm.swappiness = 1         Minimum amount of swapping without disabling it entirely.
+# vm.swappiness = 10        This value is sometimes recommended to improve performance when sufficient memory exists in a system.
+# vm.swappiness = 60        The default value.
+# vm.swappiness = 100       The kernel will swap aggressively.
+###################################################################
+vm.swappiness = 10
+</Code>
+That's about it !\\
+If you want to know what your current system's swappiness is, use ''sysctl -a | grep swappiness''.
 ===== Shell Customization & Utilities =====
@@ Line 421: / Line 505: @@
 ==== Time Sync ====
-One important aspect when setting up a new system is time synchronization, to achieve this we'll use ''ntp''. Debian 9 has time synchronization built in and activated by default using the standard ''ntpd'' time server, provided by the ''ntp'' package.
+One important aspect when setting up a new system is time synchronization, to achieve this we'll use ''ntp''. Debian 9 should have time synchronization built in and activated by default using the standard ''ntpd'' time server, provided by the ''ntp'' package.
 First lets check whether the ''ntp'' time server is running:
@@ Line 453: / Line 537: @@
 You can refer to [[https://blog.sleeplessbeastie.eu/2015/04/27/how-to-manage-system-services-on-debian-jessie/|this list of systemctl commands]] to see how to manage services.
+To configure your system's time zone use the ''dpkg-reconfigure tzdata''
 ==== SMART ====
@@ Line 540: / Line 626: @@
 </Code>
-=== Automating Actions ===
-We can run ''smartmontools'' as a system Deamon in order to have the tests run automatically, to do this we'll edit the ''smartmontools'' config file:
+=== Sending Mail ===
+It would be nice to have ''smartd'' automatically sending us emails whenever SMART detects something worth notifying.\\
+This could be achieved by installing and configuring a MTA like a ''postfix'' server.
+Although, as we don't need (want) this server to become an smtp relay, we'll rather install the ''heirloom-mailx / s-nail'' package and configure it to use an external smtp relay, see the [[https://www.systutorials.com/1411/sending-email-from-mailx-command-in-linux-using-gmails-smtp/|this article @systurials.com]].
+<Code:bash>
+> apt-get install heirloom-mailx
+> cd /usr/bin/
+> ln -s heirloom-mailx mailx
+</Code>
+We need to create the ''mailx'' symbolic link in order to have it available as such under the command line.
+As is stated in the [[https://linux.die.net/man/1/mailx|mailx man page]]:
+<WRAP box 90%>
+Normally, mailx invokes ''sendmail(8)'' directly to transfer messages. If the ''smtp'' variable is set, a SMTP connection to the server specified by the value of this variable is used instead. If the SMTP server does not use the standard port, a value of ''server:port'' can be given, with port as a name or as a number.
+</WRAP>
+Let's test an external smtp server using an all-in-one command:
+<Code:bash>
+> echo "My message body" | mailx -v \
+-s "Server Test Mail" \
+-S smtp-use-starttls \
+-S ssl-verify=ignore \
+-S smtp-auth=login \
+-S smtp=smtp.gmail.com:587 \
+-S from="*******@gmail.com(System Watch)" \
+-S smtp-auth-user=*******@gmail.com \
+-S smtp-auth-password=******* \
+recipient@domain.com
+Resolving host "smtp.gmail.com:587" ... done
+Connecting to "2a00:1450:400c:c06::6c:587" ...connected.
+</Code>
+You'll probably need to allow //Less secured apps access// in you gmail account for this to work ([[https://myaccount.google.com/lesssecureapps|here]])
+Once this works, and you effectively receive the email at the ''recipient@domain.com'' address, it's time to configure a system wide smtp configuration for ''mailx'' and send a last test.\\
+Note that, although the usage of the user specific ''~/.mailrc'' config file was quite obvious, it was much more tricky to determine the correct location (and name) of the global configuration file used by the ''mailx'' command. Having seen a lot of references to ''/etc/mail.rc'', it took a peak at the source code of ''s-nail'' to finally establish that, for v14.8.16, the correct location for the general configuration file was ''/etc/s-nail.rc''!
+<Code:bash>
+> nano /etc/s-nail.rc
+TYPE
+account gmail {
+  set smtp-use-starttls
+  set ssl-verify=ignore
+  set smtp=smtp.gmail.com:587
+  set smtp-auth=login
+  set smtp-auth-user=*******@gmail.com
+  set smtp-auth-password=*******
+  set from="*******@gmail.com(System Watch)"
+}
+> echo "Global mailx configuration file was used here." | mailx -v -A gmail -s "Sent with global configuration" recipient@domain.com
+Resolving host "smtp.gmail.com:587" ... done
+Connecting to "2a00:1450:400c:c0b::6d:587" ...connected.
+</Code>
+When this works, we're ready to automate SMART reports...
+=== Automating SMART Reports ===
+First we'll need to run ''smartmontools'' as a system Deamon in order to have SMART tests run automatically, to do this we'll edit the ''smartmontools'' config file:
 <Code:bash>
 > sudo nano /etc/default/smartmontools
@@ Line 551: / Line 699: @@
 Now, ''smartd'' will be launched at boot time.
-To define how SMART will scan the disk and what actions are to be taken if SMART returns any error:
+Next, to define how SMART will scan the disk and what actions are to be taken if SMART returns any error, we'll configure ''smartd''.\\
-<Code:bash>
+By default, ''smartd'' will run ''/usr/share/smartmontools/smartd-runner'', which will create a temporary report file, and in turn, will run scripts located in ''/etc/smartmontools/run.d/''. As we'd like to modify this behavior, well create our own copy of those scripts:
-> sudo nano /etc/smartd.conf
-...
+<Code:bash|As root do:>
-# CHECK THAT THE FOLLOWING IS UNCOMMENTED
+> mkdir -p ~/.smartd/run.d
-DEVICESCAN -a -H -l error -l selftest -f -s(S/../..././02|L/../../1/00)-m tech@tacticz.com -M exec /usr/share/smartmontools/smartd-runner...
+> cp /usr/share/smartmontools/smartd-runner ~/.smartd/
+> cp /etc/smartmontools/run.d/10mail ~/.smartd/run.d/10mailx
+> nano ~/.smartd/smartd-runner
+#!/bin/bash -e
+run-parts --report --lsbsysinit --arg="$1" \
+    --arg="$2" --arg="$3" -- /root/.smartd/run.d
+> nano ~/.smartd/run.d/10mailx
+#!/bin/bash -e
+# Send mail if /usr/bin/mailx exists
+if ! [ -x /usr/bin/mailx ]; then
+        echo "Your system does not have /usr/bin/mailx.  Install the mailx package"
+        exit 1
+fi
+echo "$SMARTD_FULLMESSAGE" | /usr/bin/mailx -A gmail -s "$SMARTD_FAILTYPE - $SMARTD_MESSAGE" $SMARTD_ADDRESS
 </Code>
-( -a) This enables some common options. You almost certainly want to use it. To check the SMART health status (-H). To report increases in both SMART error logs (-l). To check for failure of any Usage Attributes (-f) . "-s (S/../.././02|L/../../1/00)" This schedules the short and long self-tests. In this example, the short self-test will run daily at 2:00 A.M. The long test will run on every Monday at 0:00. For more information, see [[https://www.freebsd.org/cgi/man.cgi?query=smartd.conf&manpath=ports&sektion=5|the smartd.conf man page]].
+These two scripts should now work together and generate a SMART report that gets emailed to a defined email address. In order to achieve this, we still need to configure the ''smartd'' service through the ''/etc/smartd.conf file''. Open it and uncomment the first line starting with ''DEVICESCAN'', replacing it as follow:
-To test that everything works as expected, an specifically that an email gets sent to the destination address, add ''-M test'' right after the ''DEVICESCAN'' keyword and restart smartd:
 <Code:bash>
-> nano /etc/smartd.conf
+> sudo nano /etc/smartd.conf
 ...
-DEVICESCAN -M test -a -H -l error -l selftest -f -s (S/../..././02|L/../../1/00) -m tech@tacticz.com -M exec /usr/share/smartmontools/smartd-runner
+DEVICESCAN -M test -a -H -l error -l selftest -f -s (S/../..././02|L/../../1/00) -m tech@tacticz.com -M exec /root/.smartd/smartd-runner
 ...
+</Code>
+OPTIONS:\\
+  * -M test : specifies that a test run should be executed the next time the ''smartd'' service is restarted
+  * -a : This enables some common options. You almost certainly want to use it as it checks the SMART health status (-H). Reports increases in both SMART error logs (-l). To check for failure of any Usage Attributes (-f) .
+  * -s (S/../.././02|L/../../1/00) : This schedules the short and long self-tests. In this example, the short self-test will run daily at 2:00 A.M. The long test will run on every Monday at 0:00.
+For more information, see [[https://www.freebsd.org/cgi/man.cgi?query=smartd.conf&manpath=ports&sektion=5|the smartd.conf man page]].
+To test that everything works as expected, specifically that emails get sent, and since we have set the ''-M test'' option, we'll restart ''smartd''. As we restart the service, it is a good idea to have an eye on the log files so to get a feedback of the operations:
+<Code:bash: Watch logs in one terminal>
+> lnav /var/log/
+</Code>
+<Code:bash| Restart smartd from another terminal>
 > systemctl restart smartd
 </Code>
-Watching the ''var/log/deamon.log'' file will let you know if a problem occurs:
+You should see something like this in the logs:
-<Code:bash>
+<Code>
-> lnav /var/log/
 ...
-Nov 24 01:00:16 stock smartd[21081]: Your system does not have /usr/bin/mail.  Install the mailx or mailutils package
+Nov 25 03:06:09 cloud smartd[433]: Opened configuration file /etc/smartd.conf
-Nov 24 01:00:16 stock smartd[21081]: run-parts: /etc/smartmontools/run.d/10mail exited with return code 1                                                               Nov 24 01:00:16 stock smartd[21081]: Test of /usr/share/smartmontools/smartd-runner to tech@tacticz.com: failed (32-bit/8-bit exit status: 256/1)
+...
+Nov 25 03:06:09 cloud smartd[433]: Device: /dev/sda [SAT], state read from /var/lib/smartmontools/smartd.HGST_HUS726020ALA610-N4G3M6DY.ata.state
+...
+Nov 25 03:06:09 cloud smartd[433]: Monitoring 3 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices
+Nov 25 03:06:09 cloud smartd[433]: Executing test of /root/.smartd/smartd-runner to recipient@domain.com
+Nov 25 03:06:09 cloud smartd[433]: Test of /root/.smartd/smartd-runner to recipient@domain.com: successful
 ...
 </Code>
-We need to install a package to allow emails to be sent...
+Test emails should be delivered to the target mailbox.\\
+When it works, remove the ''-M test'' option in ''/etc/smartd.conf'' and restart the deamon with ''systemctl restart smartd''.
-=== Sending Mail ===
-To avoid having to configure a local ''postfix'' server, we'll install the ''heirloom-mailx'' package and configure it to use an external smtp relay, see the [[https://www.systutorials.com/1411/sending-email-from-mailx-command-in-linux-using-gmails-smtp/|whole article @systurials.com]].
-<Code:bash>
+=== FAIL2BAN ===
-> apt-get install heirloom-mailx
+On a physical server it might be a good idea to install ''fail2ban'' to establish a minimal protection.
-</Code>
+[[https://upcloud.com/community/tutorials/install-fail2ban-debian/]]
-Let's test our external smtp server using an all-in-one command:
-<Code:bash>
-> mailx -v -s "Cloud GSP2 Test Mail" \
--S smtp-use-starttls \
--S ssl-verify=ignore \
--S smtp-auth=login \
--S smtp=smtp://mail.runbox.com:587 \
--S from="root@cloud.gsp2.com(System Watch)" \
--S smtp-auth-user=thibaut@tacticz.com \
--S smtp-auth-password=H3rcul35 \
--S ssl-verify=ignore \
-thibaut.demuynck@gmail.com
-</Code>
 ===== Install VirtualBox guest additions =====