AWS SES domain verification

Amazon’s AWS SES service allows you to verify a domain so that you can then send email from any of that domain addresses through your EC2 instances.

The verification is done by adding a TXT record to your DNS server for that domain. The TXT record name looks like:

_amazonses.zonalivre.org OtNct7ugD0fOgjp70xpNpWj4K0xPcGcUopkcsiby9nE=

This is straightforward most times, however, if you host your dns externally, some dns providers don’t allow underscores in the txt record name. In these cases, your TXT record needs to be written in this format instead:

zonalivre.org amazonses:OtNct7ugD0fOgjp70xpNpWj4K0xPcGcUopkcsiby9nE=

Happy SES’ing !

AWS SES domain verification

Docker Hello World Example

Some quick notes on how to get up and running with Docker.

Installation

On Fedora this turned out to be pretty easy, simply:

dnf install docker

For other systems see the Docker installation manual.

Start docker demon

systemctl status -l docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: active (running) since Sat 2015-12-12 22:40:41 GMT; 3min 35s ago
Docs: http://docs.docker.com
Main PID: 5818 (docker)
CGroup: /system.slice/docker.service
└─5818 /usr/bin/docker daemon --selinux-enabled

Dec 12 22:40:01 asterix docker[5818]: time="2015-12-12T22:40:01.803909914Z" level=error msg="WARNING: No --storage-opt dm.thinpooldev specified, using loopback; this configuration is strongly discouraged for production use"
Dec 12 22:40:40 asterix docker[5818]: time="2015-12-12T22:40:39.995085846Z" level=warning msg="Docker could not enable SELinux on the host system"
Dec 12 22:40:40 asterix docker[5818]: time="2015-12-12T22:40:40.059743984Z" level=info msg="Option DefaultDriver: bridge"
Dec 12 22:40:40 asterix docker[5818]: time="2015-12-12T22:40:40.059788786Z" level=info msg="Option DefaultNetwork: bridge"
Dec 12 22:40:40 asterix docker[5818]: time="2015-12-12T22:40:40.388329409Z" level=info msg="Firewalld running: true"
Dec 12 22:40:41 asterix docker[5818]: time="2015-12-12T22:40:41.278276562Z" level=info msg="Loading containers: start."
Dec 12 22:40:41 asterix docker[5818]: time="2015-12-12T22:40:41.278585203Z" level=info msg="Loading containers: done."
Dec 12 22:40:41 asterix docker[5818]: time="2015-12-12T22:40:41.278606222Z" level=info msg="Daemon has completed initialization"
Dec 12 22:40:41 asterix docker[5818]: time="2015-12-12T22:40:41.278627443Z" level=info msg="Docker daemon" commit="cb216be/1.8.2" execdriver=native-0.2 graphdriver=devicemapper version=1.8.2-fc22
Dec 12 22:40:41 asterix systemd[1]: Started Docker Application Container Engine.

An interesting warning about no –storage-opt db.thinpooldev specified. This is related to Docker’s storage driver, which the docs explain in good detail.

Run Hello World

First, some info about the docker installation

docker info

It’s better to follow along the official and pretty good step by step documentation on basic docker usage, but the basics are below:

Run a single command

[root@asterix ~]# docker run ubuntu:14.04 /bin/echo 'Hello world'
Hello world

Interactive shell

[root@asterix ~]# docker run -t -i ubuntu:14.04 /bin/bash
root@8aed75e824e2:/# hostname
8aed75e824e2
root@8aed75e824e2:/# exit
exit
[root@asterix ~]#

Restart a container that was previously running

docker ps -a
docker restart $CONTAINER_ID
docker exec -it $CONTAINER_ID /bin/bash

Copy file from container to host

docker cp $CONTAINER_ID:$CONTAINER_PATH $HOST_PATH

Under the covers

Basic information about the newly created image:

[root@asterix ~]# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
docker.io/ubuntu 14.04 d55e68e6cc9c 4 days ago 187.9 MB

Local docker containers are stored in /var/lib/docker

[root@asterix ~]# ls -l /var/lib/docker/
total 36
drwx------ 5 root root 4096 Dec 12 23:09 containers
drwx------ 5 root root 4096 Dec 12 22:50 devicemapper
drwx------ 7 root root 4096 Dec 12 22:50 graph
-rw-r--r-- 1 root root 5120 Dec 12 23:09 linkgraph.db
-rw------- 1 root root 114 Dec 12 22:50 repositories-devicemapper
drwx------ 2 root root 4096 Dec 12 22:50 tmp
drwx------ 2 root root 4096 Dec 12 22:49 trust
drwx------ 2 root root 4096 Dec 12 22:40 volumes

And we can see what looks like a ext4 filesystem:

[root@asterix ~]# file /var/lib/docker/devicemapper/devicemapper/data
/var/lib/docker/devicemapper/devicemapper/data: Linux rev 1.0 ext4 filesystem data, UUID=6d11e0b5-f063-4a4f-99f4-36253b12b297 (extents) (large files) (huge files)
Docker Hello World Example

CentOS 7 kickstart file

The minimal CentOS 7 kickstart file I could come up with for my requirements. Enjoy.

install
lang en_GB.UTF-8
keyboard uk
poweroff
timezone UTC
auth  --useshadow  --passalgo=sha512
selinux --disabled
firewall --disabled
services --enabled=NetworkManager,sshd
eula --agreed
rootpw --plaintext password
ignoredisk --only-use=sda
bootloader --location=mbr --timeout=0
zerombr
clearpart --all --initlabel
part swap --asprimary --fstype="swap" --size=1024
part /boot --fstype xfs --size=200
part pv.01 --size=1 --grow
volgroup rootvg01 pv.01
logvol / --fstype xfs --name=lv01 --vgname=rootvg01 --size=1 --grow

repo --name=base --baseurl=http://mirror.centos.org/centos/7/os/x86_64
repo --name=updates --baseurl=http://mirror.centos.org/centos/7/updates/x86_64
url --url="http://mirror.centos.org/centos/7/os/x86_64"

%packages --nobase --ignoremissing
@core
augeas
%end


%post --log=/root/postinstall.log

# Do the bare minimum so that I can ssh to the box and run a shell script to execute whatever provisioning needs happening

mkdir -p /root/.ssh
chmod 700 /root/.ssh
echo "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6hE75Ox6wDfXVJzXeKdyUBO4o19TtGxboJTI2vR3CE9ZJbODIxSr+tfMZcwmuSF892PiahhVzAA2wJ6LdMtFH6FUIGvjU0i7jIo/x+TmvheH46N9qllo2C2ZlxL/HbpRYIyqEntUYcBQzYBvUwnzoDFgS1GhG4LalYp0U9zlHGOA/Wk7qBjH8Ca1mtPSnxudsb/NwERIjfLbvdX9Fc+vkx6fs3ykJv+p8lPEZkw3kcVAfuyhnXzE7kprSHDuOuQo0FDvCTjy9ISxZPvExKT7bD7vQRlrx9PLzYSWI7/evonWHR8c/jPS8U56ii8YH/rtC/iqo4LiwKFxoxaDdS2wD joaocosta@zonalivre.org" > /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys


augtool -s <<EOF
#root login needs to be enabled during initial setup so the project specific scripts can be executed
set /files/etc/ssh/sshd_config/PermitRootLogin yes

#This saves time during vm startup
set /files/etc/grub.conf/timeout 0

#Removed because otherwise user install scripts can't use sudo
rm /files/etc/sudoers/Defaults[requiretty]
EOF

%end
CentOS 7 kickstart file

qemu disk format autodetection

Lets say you use qemu like this:

sudo qemu-system-x86_64 \
    -hda system.raw	\
    -hda data.raw

In recent versions of qemu, you might see this error message:

WARNING: Image format was not specified for 'system.raw' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

To avoid the warning and remove the restriction on block 0 write operations, do what the error message suggests and explicitly set the disk file format to raw, ie:

sudo qemu-system-x86_64 \
    -drive file=system.raw,format=raw,index=0,media=disk \
    -drive file=data.raw,format=raw,index=1,media=disk

Happy qemuing!

qemu disk format autodetection

Choosing a domain registrar

For a long time I’ve registered my domains with godaddy. They were recommended by a friend back in the early 00’s, and they worked well enough, but there have always been a few gotchas that made using the service unpleasant:

  • Heavy UI, required more work than needed to get things done. To be fair, they have improved this over the time.
  • Constant up selling of things I don’t need.
  • Bait tactic of charging for a cheap one year domain, only to be hit with a high renewal fee the year after.

Still, for a relatively light domain registrar user like myself, those kinks weren’t really enough to justify the trouble of moving.

Until recently a couple of my domains came up for renewal, and I finally decided it was time to try something different. My first port of call was reddit.com, and as usual, they didn’t disappoint. The site is packed full of threads discussing domain registrars. Out of these, a gem came out.

https://domcomp.com/ is a service that allows you to compare different domain registrars. It’s pretty cool, if you are shopping for a domain registrar, you should definitely check it out.

I like the ability to compare prices for different .TLDs and purchase options (New, Renew, Transfer), while the discussions on reddit gave me a general idea of what kinds of features/service levels to expect.

After some research, I’ve narrowed it down to either namecheap or namesilo and will transfer a domain or two to try them out.

Choosing a domain registrar

Redirect STDOUT/STDERR

Sometimes you want to redirect your shell script’s STDOUT/STDERR to a log file. This is pretty simple to do, ie:

./myscript.sh &> output.log

But sometimes you want to do the redirect inside the script itself ( ie, maybe redirecting is conditional, or maybe the file name is conditional on something only the script knows about)


#!/bin/bash

exec >> output.log
exec 2>&1

>&2 echo "This was printed to stderr"
echo "This was printed to stdout"

Or if you want to write to both a file and the terminal:

#!/bin/bash

exec 1> >(tee -a output.log)
exec 2>&1

>&2 echo "This was printed to stderr"
echo "This was printed to stdout"

Stackoverflow has other ways to do it:

Redirect STDOUT/STDERR