Setup external USB disk as NTFS volume on Raspbian

I intend to use an external 2.5″ USB disk formatted as NTFS volume on my Raspberry Pi. Since its rather larger (5TB) I don’t want to use MBR but GPT instead. Here’s a short list of commands I’ve used to setup the disk.

Start by identifying the connected disk:

> lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  4.6T  0 disk
└─sda1        8:1    0  4.6T  0 part
mmcblk0     179:0    0 14.9G  0 disk
├─mmcblk0p1 179:1    0  1.5G  0 part
├─mmcblk0p2 179:2    0    1K  0 part
├─mmcblk0p5 179:5    0   32M  0 part
├─mmcblk0p6 179:6    0   69M  0 part /boot
├─mmcblk0p7 179:7    0  8.2G  0 part /
├─mmcblk0p8 179:8    0  512M  0 part
└─mmcblk0p9 179:9    0  4.5G  0 part

My disk is sda.

I now usually used fdisk as a partitioning tool. However, there’s a tool I can highly recommend. Its called parted and can be installed using:

sudo apt-get install parted

Since I’ll want to use ntfs as file system, I’ll need to install the ntfs drivers:

sudo apt-get install ntfs-3g

Now create a new GPT partition table:

> sudo parted /dev/sda mklabel gpt
Warning: The existing disk label on /dev/sda will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
Information: You may need to update /etc/fstab.

Now create a new partition with ntfs. I’ll use all of the available space, so its from 0 to 100%:

> sudo parted -a opt /dev/sda mkpart primary ntfs 0% 100%
Information: You may need to update /etc/fstab.

Now format the disk in quick format with ntfs. It will label the partition as “SynoBackups”:

> sudo mkfs.ntfs -L SynoBackups -Q /dev/sda1
Cluster size has been automatically set to 4096 bytes.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.

This label is very helpful in identifying the partition, even when it is connected to a different USB port. Using a device like sda might point to a different drive, so its better to use the label. This is one of the big advantages of using gpt in comparison to mbr.

Let’s see the label in action:

sudo lsblk --fs
NAME        FSTYPE LABEL       UUID                                 MOUNTPOINT
sda
└─sda1      ntfs   SynoBackups 4EE12D1B5321171F
mmcblk0
├─mmcblk0p1 vfat   RECOVERY    525E-19E4
├─mmcblk0p2
├─mmcblk0p5 ext4   SETTINGS    ceb0ae64-8675-406b-8eed-2244c26814c8
├─mmcblk0p6 vfat   boot        8454-E385                            /boot
├─mmcblk0p7 ext4   root0       65678398-7f53-48ec-8452-c277500fb4e8 /
├─mmcblk0p8 vfat               AC0D-3FB1
└─mmcblk0p9 ext4               ff645116-fe34-43bf-a580-b89fa963085d

Note that there’s also a more specific id, the UUID. We will use this UUID later when we configure a default mount point in /etc/fstab.

Now we’ll try to mount the new partition. Create a folder to mount the partition and mount it manually:

sudo mkdir /mnt/backups
sudo mount -o defaults /dev/sda1 /mnt/backups

Verify that the disk is mounted and try to write some stuff to it:

> df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       8.0G  2.7G  4.9G  36% /
devtmpfs        457M     0  457M   0% /dev
tmpfs           462M     0  462M   0% /dev/shm
tmpfs           462M  6.2M  455M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           462M     0  462M   0% /sys/fs/cgroup
/dev/mmcblk0p6   68M   23M   46M  33% /boot
/dev/sda1       4.6T  210M  4.6T   1% /mnt/backups

> sudo lsblk --fs
NAME        FSTYPE LABEL       UUID                                 MOUNTPOINT
sda
└─sda1      ntfs   SynoBackups 4EE12D1B5321171F                     /mnt/backups
mmcblk0
├─mmcblk0p1 vfat   RECOVERY    525E-19E4
├─mmcblk0p2
├─mmcblk0p5 ext4   SETTINGS    ceb0ae64-8675-406b-8eed-2244c26814c8
├─mmcblk0p6 vfat   boot        8454-E385                            /boot
├─mmcblk0p7 ext4   root0       65678398-7f53-48ec-8452-c277500fb4e8 /
├─mmcblk0p8 vfat               AC0D-3FB1
└─mmcblk0p9 ext4               ff645116-fe34-43bf-a580-b89fa963085d

> echo "success" | sudo tee /mnt/backups/file
success
> cat /mnt/backups/file
success
> rm /mnt/backups/file
> sudo umount /mnt/backups

Now we’ll add the partition to /etc/fstab so that it can be mounted automatically:

UUID=4EE12D1B5321171F   /mnt/backups    ntfs    defaults        0       2

See that I’m now using the UUID instead of /dev/sda to mount the ntfs volume to /mnt/backups. We can test the new setting:

> sudo mount -a
> df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       8.0G  2.7G  4.9G  36% /
devtmpfs        457M     0  457M   0% /dev
tmpfs           462M     0  462M   0% /dev/shm
tmpfs           462M  6.2M  455M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           462M     0  462M   0% /sys/fs/cgroup
/dev/mmcblk0p6   68M   23M   46M  33% /boot
/dev/sda1       4.6T  210M  4.6T   1% /mnt/backups

I think this is a really nice change in mounting the volumes and will create a more stable configuration, regardless which USB port you’ve used to connect your drive.

Block SSH connections by origin of IP address

If you’re exposing services to the internet, you’ll notice a lot of connection attempts. To block those bots and scripts trying to login to your machine, you should use fail2ban.

However, you can also limit the range of allowed origins of the IP addresses. The company MaxMind provides a database of IP addresses and their origin contries. You can configure your machine in such a way that only certain country codes are allowed.

Start by installing the geoip client and database by using this apt command:

sudo apt-get install geoip-bin geoip-database

This database is updated automatically, when you’ve got your machine configured for auto updates.

The next step is to save this script to your machine in /usr/local//usr/local/bin/ipfilter.sh:

Edit the script to your needs, e.g. by limiting the number of allowed countries. Now make this script executable:

chmod +x /usr/local/bin/ipfilter.sh

It is time to test it. Try the command with a known IP in America and one from a local network or known IP from the allowed countries:

> /usr/local/bin/ipfilter.sh
Usage:  ipfilter.sh <ip>
> /usr/local/bin/ipfilter.sh 8.8.8.8
> echo $?
1
> /usr/local/bin/ipfilter.sh 192.168.1.1
> echo $?
0

Notice the different exit codes of the script. If the IP is from a country that is allowed or if it is from a local network, it will exit with 0, otherwise 1. We can use this script now to configure a filter for sshd in the /etc/hosts.allow and /etc/hosts.deny files.

Add to /etc/hosts.allow:

sshd: ALL: aclexec /usr/local/bin/ipfilter.sh %a

As documented in the hosts_options(5) man page, the standard output is redirected to /dev/null, so that there’s no chance for you to get the output from echo. And as you want the exit status to be taken into account, you should use aclexec instead of spawn. Indeed the man page says for aclexec: “The connection will be allowed or refused depending on whether the command returns a true or false exit status.”

https://unix.stackexchange.com/a/149057/298669

I’ve previously used “spawn” instead of “aclexec” but the IPs weren’t blocked. There were still connection attempts in the fail2ban log. By using aclexec, the exit code will be properly used for filtering.

Add to /etc/hosts.deny:

sshd: ALL

Please note the trailing newline. If this is the last entry in the hosts file, you’ll need to add a newline. Otherwise the role won’t be active.

Do a reboot of your machine and try to connect. You should still be able to connect 😉 Otherwise you’ll need to revert this changes locally, since you’ve successfully blocked yourself from accessing that machine.

You can have a look at the /var/log/auth.log and will see entries like this for example (blocking an IP from China/CN):

Sep 30 12:10:32 raspberrypi root: DENY sshd connection from 222.186.30.76 (CN)
Sep 30 12:10:32 raspberrypi sshd[886]: aclexec returned 1
Sep 30 12:10:32 raspberrypi sshd[886]: refused connect from 222.186.30.76 (222.186.30.76)

This should reduce the amount of blocked SSH connections attempts significantly, if configured to a smaller selection of countries.

Email notification for fail2ban events

So I’ve configured my fail2ban installation and I’m also able to send emails. But wouldn’t it be awesome if I’ll get notified via email about any fail2ban event?

We start with editing the /etc/fail2ban/jail.local file. Look for the destemail and action parameters and change them accordingly:

mta = sendmail
destemail = recipient@domain.name
senderemail = sender@domain.name
action = %(action_mwl)s

The action can be one of these, whereby I’ve chosen action_mwl:

  • action_: ban only the IP
  • action_mw: ban the IP and send email with whois information about the banned IP
  • action_mwl: ban the IP and send email with whois information about the banned IP and add relevant log lines to the email
  • action_cf_mwl: notify Cloudfare about the offending IP, ban the IP and send email with whois information about the banned IP

Do a restart of fail2ban:

sudo systemctl restart fail2ban

You’ll receive a lot of emails from fail2ban. This also includes any starts and stops of fail2ban as well as the ban notifications. You can limit this behavior by adding following content to the file /etc/fail2ban/action.d/mail-buffered.local:

[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart =

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop =

Now copy this file a few times with different file names:

sudo cp /etc/fail2ban/action.d/mail-buffered.local /etc/fail2ban/action.d/mail.local
sudo cp /etc/fail2ban/action.d/mail-buffered.local /etc/fail2ban/action.d/mail-whois-lines.local
sudo cp /etc/fail2ban/action.d/mail-buffered.local /etc/fail2ban/action.d/mail-whois.local
sudo cp /etc/fail2ban/action.d/mail-buffered.local /etc/fail2ban/action.d/sendmail-buffered.local
sudo cp /etc/fail2ban/action.d/mail-buffered.local /etc/fail2ban/action.d/sendmail-common.local

Do a restart of fail2ban:

sudo systemctl restart fail2ban

You should now only receive emails for ban events.

Protect SSH services with fail2ban

If you’ll open SSH on a server to the open internet, you’ll notice a lot of bots trying to login. You certainly should setup certificate based login, but banning offending IPs is also an important security measure.

I’ve installed fail2ban on my Raspbian installations and want to explain the installation and configuration. Its quite easy and the benefits are huge!

sudo apt-get install fail2ban

Create a copy of the original configuration file so that it won’t be overwritten by any updates:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Search for a block for [default]. You should set:

bantime = 10m
findtime = 10m
maxretry = 5

These are the general settings. The settings for sshd should be a little bit stricter. Search a block for [sshd]. You should set:

enabled = true
maxretry = 3

You can enable and start fail2ban now using systemctl:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Verify its up and running:

sudo systemctl status fail2ban.service
sudo fail2ban-client status
sudo fail2ban-client status sshd

If you end up being locked out, you can unlog an offending IP address using this command:

sudo fail2ban-client set sshd unbanip <offenders IP>

Banned connections will be dropped immediately by the firewall and should be visible with a “connection refused”.

Configure mail transport agent on Raspbian with external SMTP server

I want to get email notifications for actions on my Raspberry Pi using Raspbian. You could setup a separate mail server for that action but that seems to be a little bit overkill.

msmtp is a mail transfer agent which uses a configured smtp server for email transfer. This allows you to send emails via a configured smtp server (in my case from my webspace provider All-Inkl.com – by creating a new account using this link you’ll support the costs for running this blog).

Upgrade your raspbian:

sudo apt-get update && sudo apt-get upgrade

Install msmtp:

sudo apt-get install msmtp msmtp-mta mailutils

Get the location of the configuration files:

> msmtp --version
msmtp version 1.6.6
Platform: arm-unknown-linux-gnueabihf
TLS/SSL library: GnuTLS
Authentication library: GNU SASL
Supported authentication methods:
plain scram-sha-1 external gssapi cram-md5 digest-md5 login ntlm
IDN support: enabled
NLS: enabled, LOCALEDIR is /usr/share/locale
Keyring support: none
System configuration file name: /etc/msmtprc
User configuration file name: /home/pi/.msmtprc

Copyright (C) 2016 Martin Lambers and others.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Configure the system configuration:

sudo vi /etc/msmtprc

The content of my configuration file (note the necessary changes for servers and email addresses):

# Set default values for all following accounts.
defaults

# Use the mail submission port 587 instead of the SMTP port 25.
port 465

# Always use TLS.
tls on
tls_starttls off

# Set a list of trusted CAs for TLS. The default is to use system settings, but
# you can select your own file.
tls_trust_file /etc/ssl/certs/ca-certificates.crt

# If you select your own file, you should also use the tls_crl_file command to
# check for revoked certificates, but unfortunately getting revocation lists and
# keeping them up to date is not straightforward.
#tls_crl_file ~/.tls-crls

# Mail account
# TODO: Use your own mail address
account user@domain.name

# Host name of the SMTP server
# TODO: Use the host of your own mail account
host <your Username provided by KAS>.kasserver.com

# As an alternative to tls_trust_file/tls_crl_file, you can use tls_fingerprint
# to pin a single certificate. You have to update the fingerprint when the
# server certificate changes, but an attacker cannot trick you into accepting
# a fraudulent certificate. Get the fingerprint with
# $ msmtp --serverinfo --tls --tls-certcheck=off --host=smtp.freemail.example
#tls_fingerprint 00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33

# Envelope-from address
# TODO: Use your own mail address
from user@domain.name

# Authentication. The password is given using one of five methods, see below.
auth on

# TODO: Use your own user name fpr the mail account
user <The username of the email account you use for sending emails>

# Password method 1: Add the password to the system keyring, and let msmtp get
# it automatically. To set the keyring password using Gnome's libsecret:
# $ secret-tool store --label=msmtp \
#   host smtp.freemail.example \
#   service smtp \
#   user joe.smith

# Password method 2: Store the password in an encrypted file, and tell msmtp
# which command to use to decrypt it. This is usually used with GnuPG, as in
# this example. Usually gpg-agent will ask once for the decryption password.
#passwordeval gpg2 --no-tty -q -d ~/.msmtp-password.gpg

# Password method 3: Store the password directly in this file. Usually it is not
# a good idea to store passwords in plain text files. If you do it anyway, at
# least make sure that this file can only be read by yourself.
# TODO: Use the password of your own mail account
password <The password of the email account you use for sending emails>

# Password method 4: Store the password in ~/.netrc. This method is probably not
# relevant anymore.

# Password method 5: Do not specify a password. Msmtp will then prompt you for
# it. This means you need to be able to type into a terminal when msmtp runs.

# Set a default account
# TODO: Use your own mail address
account default: user@domain.name

# Map local users to mail addresses (for crontab)
aliases /etc/aliases

This file contains a username and password. Therefore limit its access to only root:

sudo chmod 600 /etc/msmtprc

Duplicate the config file to ~/.msmtprc if you want to provide email configuration for your user as well.

Now configure the recipients for your systems users by setting the recipients in /etc/aliases. Make sure, that you don’t have trailing spaces behind the email addresses:

root: user@domain.name
default: user@domain.name

Let your computer now that msmtp should be used as replacement for sendmail by adding this content to /etc/mail.rc

set sendmail="/usr/bin/msmtp -t"

Test your configuration by sending an email from the terminal:

echo "Content of your mail" | mail -s "Subject" user@domain.name