Let's Encrypt with SaltStack

Dec 7, 2015devops

Let’s Encrypt aims to provide everyone a free, automated, and open SSL/TLS solution to secure the Internet. Here are some other free or low-cost Certificate Authority(CA) compared to Let’s Encrypt:

CAcert is also free, but the root certificate is NOT trusted by most browsers.
StartSSL is free for non-commercial use only, and revocations carry a handling free of US $24.90.
The commercial DV certificates usually cost from $1.99([WoSign]) to$ 9.00(Comodo SSL) annually.

Let’s Encrypt also tries to simplify the SSL certificate enrolling process with one liner command. letsencrypt-auto. Based on your server setup, you may choose any of the following options(note: letsencrypt-auto MUST run in the production server):

apache plugin will automate obtaining and installing certs for apache web server.
standalone plugin will bind to port 80 and 443 for acme(see spec) validation, so the production server MUST temporarily stop to facilitate.
webroot plugin will create a temp file as one-time token for acme validation.

Let’s encrypt, manually

The webroot plugin seems a sane approach to me, and it is supposed to be pretty straightforward according to the documentation:

$ git clone https://github.com/letsencrypt/letsencrypt
$ cd letsencrypt
$ ./letsencrypt-auto --help
$ ./letsencrypt-auto certonly --webroot -w /var/www/kunxi/www -d kunxi.org -d www.kunxi.org

Under the hood, letsencrypt-auto instantiate a virtualenv at $HOME/.local/share/letsencrypt, succeeded by pip install letsencrypt, then it offload the heavy lifting to the bin/letsencrypt in the virtualenv.

The last command exposes a temporarily URI as one-time token, such as: /.well-known/acme-challenge/_-so7o7rCJZAHjqwZTzVMk6guh83P0vyKkKQBb5mxGc. If the URL is accessible publicly, the domain is then validated, You may find the private key, privkey.pem and the fully chained cert, fullchain.pem in /etc/letsencrypt/live/kunxi.org/. And we can use them in the nginx server context as:

ssl_certificate         /etc/letsencrypt/live/kunxi.org/fullchain.pem
ssl_certificate_key     /etc/letsencrypt/live/kunxi.org/privkey.pem

Troubleshooting

The validation may fail mainly due to the deny access of hidden files, see issues #221. It is generally regarded a best practice in nginx though:

location ~ /\. {
    deny all;
}

If you accidentally leave the sensitive data, such as .git, .htpasswd in the web server, at least it is not exposed to the Internet. The workaround is quite simple:

location ~ ^/.well-known/.*$ {
    allow all;
}

location ~ /\. {
    deny all;
}

Just put the regex which matches the .well-known directory specifically before the catch-all hidden file regex. According to the nginx doc, the rules are:

nginx first searches for the most specific prefix location given by literal strings regardless of the listed order.
Then nginx checks locations given by regular expression in the order listed in the configuration file. The first matching expression stops the search and nginx will use this location. In our case, the regex ^/.well-known/.*$ in the preceding order will apply.
If no regular expression matches a request, then nginx uses the most specific prefix location found earlier.

Let’s encrypt with SaltStack

letsencrypt-auto is pretty handy for one-time SSL cert deployment, automate the SSL cert management will be more desirable from the DevOps perspective. Let’s get the following task automated via SaltStack:

Install Let’s Encrypt
Obtain a SSL cert on demand
Renew the cert monthly

Install Let’s Encrypt

Since letsencrypt-auto installation is kind of leaking, I will take the pip and virtualenv approach:

letsencrypt:
  virtualenv.managed:
    - name: /opt/letsencrypt
  pip.installed:
    - bin_env: /opt/letsencrypt
    - require:
      - virtualenv: letsencrypt

With the above snippet, we will create a virtualenv at /opt/letsencrypt, and then install letsencrypt python package with pip install letsencrypt in the above virtualenv specified by bin_env. letsencrypt is used to identify both the virtualenv and pip state, it is merely a personal preference to limit global names.

Obtain a SSL cert on demand

We also introduce two implicit dependency:

the nginx config for kunxi.org depends on the the SSL cert
the SSL cert depends on the command letsencrypt output

They are modeled as:

/etc/nginx/sites-enabled/kunxi.conf:
  file.managed:
    - source: salt://sites/kunxi/nginx.conf
    - require:
      - pkg: nginx
      - cmd: letsencrypt-kunxi.org
    - watch_in:
      - service: nginx

letsencrypt-kunxi.org:
  cmd.run:
    - name: >
             /opt/letsencrypt/bin/letsencrypt certonly --webroot
             -w /var/www/kunxi/www -d kunxi.org -d www.kunxi.org
    - creates: /etc/letsencrypt/live/kunxi.org/fullchain.pem
    - require:
      - pip: letsencrypt

The trick is the creates attributes of the cmd state(see doc): the specified command only run if the file specified by *creates` does not exist. Therefore, the SSL cert is only obtained once.

Renew the cert monthly

SaltStack also supports scheduling a cron job as follows:

renew-cert-for-kunxi.org:
  cron.present:
    - name: >
         {{ pillar.letsencrypt.venv }}/bin/letsencrypt certonly --webroot --renew
         -w {{ site_root }} -d kunxi.org -d www.kunxi.org
    - identifier: renew-cert-for-kunxi.org
    - daymonth: 1
    - hour: 10
    - minute: 0
    - require:
      - cmd: letsencrypt-kunxi.org

The cert will be renewed on first day of the month ten o’clock every month. Please read the doc carefully about the default values and the nifty identifier attribute.