A few years ago, my team and I invested the effort to build a continuous integration environment. We use TeamCity and we are quite happy with it. Once you have a platform for automated builds, you quickly become interested in automated deploys. Embarking on the painful journey to get your software to “1-click” production deploys is one of the healthiest investments you can make in your dev infrastructure.
These days we are a ways past simple deploys. Our CI infrastructure runs tests, deploys code, migrates databases, deploys infrastructure, and even moves datasets around our environments. All this good automation is glued together with various build scripts–some in python, some in bash.
If you are like us, you have multiple products running in multiple environments. Alpha, beta, uat deploys for each product, potentially hosted in multiple geographic regions in clustered deployments. All these different environments mean different configurations need to be selected at deploy time. We use a variety of methods to tackle this. On our .NET apps, we use msbuild configurations to select the correct *.config transformations. In our django apps, we use a homegrown system that will select settings.py files based on the host the code is running on.
We also need to be able to deploy different Apache configurations depending on the target environment. One of our requirements (maybe goal is a better word) is that we should be able to deploy code to a new environment without any pre-configuration, assuming the correct infrastructure is in place. If I need to throw out a new instance of our app to demo a feature change to a customer, it should be a matter of copying the build config in TeamCity, changing the name of the instance, and firing it off.
All this is just background information for me to share a cute little tip with you. When we need to deploy a different Apache config for a different environment, we use sed to make a quick and dirty mustache style templates. Sed (stream editor) is an old standby in the bearded unix hacker’s toolkit. It will parse a text stream, executing regex string replacements, and stream out the result. By defining a standard token format, in this case {{mustache style}} tokens, we can use sed to transform our Apache configs as a lightweight templating system.
Example template (named default_template) is bundled with source:
<VirtualHost *:80>
ServerName {{HOSTNAME}}:443
Alias /static {{DEPLOY_PATH}}/static
<Directory {{DEPLOY_PATH}}/static>
Order allow,deny
Allow from all
Options -Indexes
</Directory>
WSGIScriptAlias / {{DEPLOY_PATH}}/django.wsgi
</VirtualHost>
Execute transformation during deploy using sed and copy into place:
#!/bin/bash
CONFIG_NAME=$1
HOSTNAME=$2
DEPLOY_PATH_SED="\/var\/www\/$CONFIG_NAME"
DEPLOY_PATH=/var/www/$CONFIG_NAME
APACHE_PATH='/etc/apache2'
sed -e "s/{{HOSTNAME}}/$HOSTNAME/g; s/{{DEPLOY_PATH}}/$DEPLOY_PATH_SED/g" default_template > $CONFIG_NAME
cp $CONFIG_NAME $APACHE_PATH/sites-available/
ln -s $APACHE_PATH/sites-available/$CONFIG_NAME $APACHE_PATH/sites-enabled/$CONFIG_NAME
apache2ctl graceful
These are just snippets of our full deploy script and Apache template, but you get the idea. A few small points about this technique:
- We use named based virtual hosts to run multiple dev environments on one server.
- You can chain together multiple replacements using semi-colons.
- You must be careful with escaping your backslashes. In my scripts, I have to define separate variables for the sed replacement and the file copy.
TeamCity simply runs sh deploy.sh alpha alpha.fzysqr.com and everything drops into place.
Yay devops!