Now that I’ve discussed the advantages of separating your build and deployment scripts, lets look a little more closely at deployment scripts. As with build scripts, the first principle is simple and comprehensive - make sure you can deploy your product with one command. I don’t care whether it’s launched with a button on a web page, a shortcut on your desktop, or via the command line. However, you do it, the deployment script should do everything. It should take the build files, wherever you have stored them, and make that build available to your customers.

As such, the deployment script really cares about two inputs - what build to deploy, and which customers to deploy it to. The build should be easily specified by version number (and product name, if multiple products can use the same deployment script). All builds should be stored on a file server so they can be easily accessed without rebuilding them. All configurations should be available there, and a given deployment will typically take a certain configuration, depending on the deployment target and the purpose of the deployment, but it’s also worth allowing this to be specified when running the script. For deploying developer builds internally, the deploy script should be able to take files built by a developer on their own machine. That’s one reason to make sure your official build scripts and those used by developers are one and the same.

You should be able to specify the customers who receive the build as deployment targets. Deployment targets may include the developers machine, the QA team, internal dogfood, alpha testers, beta testers, and official releases. For a product like FogBugz (or Kiln) that is available in both a hosted environment as well as a licensed installer, each of those is a possible deployment target. For example, a developer may want to deploy the build he created after fixing a bug to his own machine, or to the QA team for test verification. The official build machine will be able to run the scripts that deploy an official build to alpha, beta, or regular customers.

Depending on the deployment target and the specific requirements of the product, the deployment scripts may do a large variety of things. It doesn’t really make sense for deployment scripts to be written in a build-oriented technology like make, ant, or MSBuild. I suspect that the real reason so many make replacements have been built using dynamic languages is because build managers haven’t properly separated build and deploy scripts. If they had, the advantages of being able to write code into your build scripts largely go away. The need for code (and the libraries you get with most languages) is most apparent in the deployment scripts.

For example, deployment scripts will often need to do much more than just copying files around. They may be required to setup web servers, modify databases, configure registry or xml-based settings, upload or download files to servers, create and modify web pages, change file permissions, etc. So you’ll want to use a language and framework that makes these types of operations easy to perform. Python, Powershell, and Perl seem like good choices.

Finally, as with build scripts, work to not repeat yourself in your deployment scripts. If different deployment targets use largely the same operations, refactor to eliminate duplicate code. Keeping your deployment scripts separated from your build scripts, clean, and capable of all the deployment possibilities you need with one command will make your life much easier as a build manager.

The value of setup scripts for your development and build machines has come up before. Creating these scripts can save your organization hundreds or thousands of hours of work over the course of a year. But what should they include?

Install VCS

The very first thing your setup scripts should do is install (or verify the installation of) the VCS client you use. This will be used for all of the other steps that follow, primarily as a source for getting the scripts, code, and other files needed to do all of the other things. You may need multiple VCS clients if your product uses code from open source projects, or uses a CVCS for large binaries and DVCS for code.

Common build tools repository

A minimal setup script would just install the VCS and checkout the code for the product. A more complex one would checkout the common build tools, which includes a more complete dev machine setup script, that it would then run. This script could be run in existing dev machines to verify that everything needed is available, and to setup new tools as they become needed by product development. The build tools repository could also include any internal tools used across products, copies of external build tools that don’t require any installation, and meta-information about products and repositories.

Checkout source code

After getting the VCS client, the script should automatically checkout the source code. If you have one product that is contained in a single repository, that will make this easy. If, on the other hand, the are multiple products, lots of shared code, and developers only want to work on the subset of the code they’re assigned to, it makes sense to allow them to only checkout the code they care about.

Checkout any dependent source code

The build tools should have a way of defining dependencies between the repositories used to build your products, and use this knowledge to checkout any other code needed for builds to work without any additional setup by the developer.

Install all build tools and frameworks

And I mean all of them. Install compilers, frameworks, and everything else necessary to build. Install all of these tools at standard locations. As much as possible use silent installers that don’t require any interaction. If user interaction is required, run those installers first. Of course, you should only do this for tools that cannot be provided via your VCS.

Configure OS to make development possible/easy

If your build scripts require administrator access to run in non-interactive mode, turn off UAC on windows machines. If you need to make sure any OS features are installed (IIS, anyone?), make sure that happens also.

Setup any hard dependencies (environment, path, DBs, registry, etc.)

After getting the OS into the correct state, it’s time to take care of hard dependencies that your build scripts still need (you’ve tried to eliminate most of those, right?). Create databases, set registry values, add directories to the PATH, and set other environment variables. This is the time to do it all, and once it’s done, a new developer should be able to run any build script and have it succeed.

Convenience items

There are other things that your setup script can do that add convenience but aren’t required for building the product. For example, it could setup shared folders on the dev box, to make it easier to share files between developers. If there are dev tools that most developers at your organization use, but that aren’t required to build the code, it’s worth adding them as an optional item in the dev machine setup script. If pair programming is a key part of your culture, this may be more important, approaching the level of a requirement, a common machine setup that is interchangeable. Another useful addition would be setting up virtual machines that the developers can use for sandboxed testing.

To be complete, I’ll mention a couple things that would otherwise go without saying. This setup script should be smart about any of these steps that have already been done, and skip those steps both in order to run faster and also to avoid breaking something. Developers will want the ability to rerun this script when it changes, so it should work well in that case. If necessary, break it up into smaller, more focused scripts, or add command line parameters to do a subset of the work. As you create this script, start small, build incrementally. Trying to get every situation and configuration right the first time through is a sure path to never finishing it. But make sure that when you first release it to your team that it actually saves some work, at least enough to be worth the hassle of remembering to use the new tool. Once it’s available, the developers on your team will become contributors to this script.

I got a couple of comments on Hacker News regarding my post about eliminating absolute paths in your code and scripts. User fragmede brought up the issue of using absolute paths to a common bin folder so that once you’ve built you can easily run and test your code. I responded by pointing out that what he actually is discussing is not part of a build script, but part of a deploy script. It’s easy to let the two scripts ambiguously coincide or put stuff that belongs in one in the other, but there is a lot of value in keeping them separate.

First of all, you don’t always want to deploy when you build, even on a development machine. On the build machine this concept is more clear. You may have a continuous build machine, which rebuilds your product from source with every checkin, or on a regular schedule. These builds could potentially be deployed to the same machine, but you’ll more likely want them deployed elsewhere, if at all. It may \be that all the continuous build is needed for is to verify that there are no build breaks or test failures, and no deployment is necessary.

Second, building for every deployment is wasted effort. Once you’ve built your product in a given configuration, you should be able to deploy what was built to a number of different locations. Obviously, you should be able to deploy it locally, to your dev machine. But if you’ve built on an official build machine, you’ll also want to be able to deploy to an internal dogfood. If your product is web based, this probably means configuring and setting up the internal server with the built files. If it’s a client app, it means making that client app available to everyone in your organization by notifying them and putting the installer on an internal file share. You may also want to deploy the build, once its been thoroughly tested, to beta testers and finally to all of your customers.

The last two points highlight an overriding principle: you want to decouple build and deployment scripts so you can change each separately. For FogBugz and Kiln there are lots of different build configurations - ship/debug, x86/x64, hosted/licensed, etc. Likewise there are lots of possible deployment targets - dev machine, QA machine, licensed installer available on internal share, or available as a customer download, internal dogfood installation, and obviously, deployment to actual customers. You want the capability of deploying any given configuration to any given deployment target for testing/debugging purposes, though of course, you’ll rarely if ever use certain configuration/target combinations.

Of course, the normal script that developers will run should wrap these scripts, so that they don’t have to run the build script, then deploy the build to their dev box in two separate steps.

RT @apowter: Less than two weeks in to our new @FogBugz installation & we just hit triple-digit bug count! Loving how easy it is to file …

In the last few posts I’ve discussed the hard dependencies you want to eliminate from your build scripts - things like absolute paths and databases. It’s quite natural to ask why I used the term “hard dependencies”, and what a “soft” dependency would look like. Soft dependencies are basically any thing that you can get from your VCS or that is setup automatically when you create a new development or build machine. Let’s look at some examples.

Relative paths

In the post on absolute paths I mentioned the value of relative paths. These are typically a soft dependency because they point to either 1) code or other files that were retrieved from the VCS or 2) build tools or other programs initialized with a dev environment setup script.

Source code and files from other repositories

Besides the paths themselves, the files you get from other repositories or from other locations in the same repository are soft dependencies. You obviously need them for your build scripts to work, but you basically get them for free. If the source and other files in question come from a different repository in your VCS, you should use a tool like svn:externals or hg subrepos to make sure that code is available on your machine before building.

Any hard dependency for which there is a robust fallback behavior

For all of the hard dependencies mentioned here, you can change them to be soft dependencies by having a robust fallback behavior. This means that if the dependency is not available for whatever reason, the build script can continue unaltered and still succeed using an alternative method for building the code that is equally valid. Of course, if that is the case, you should probably eliminate the hard dependency altogether. This stage can be an important step on the way from having lots of hard dependencies in your scripts to few or none.

Hard dependencies that are setup automatically with a dev machine setup script

Dev machine setup scripts are not ideal (for more on the ideal, keep reading), but they can be very useful when you’ve got some tough hard dependencies for which it really doesn’t make business sense to use one of the methods already listed for turning it into a soft dependency. By using a script to setup these hard dependencies automatically for dev machines, you make it much easier for new developers (and experienced devs on new machines) to get up and coding quickly. Your build scripts now have an implied dependency (i.e. that the machine setup script has been run), but that one dependency can cover a whole range of issues.

Ideal build system

The ideal build system will have no hard dependencies. So what will it look like? Well, you’ll still need to get the code onto your dev machine, but you won’t need a dev machine setup script. So basically, you’ll just:

1. Install the VCS client
2. Checkout your code
3. Run the build script

This ideal build system will not require you to install any development tools. They’ll be available at relative paths inside the repository you checked out (or a common build tools repository included via svn:externals or hg subrepositories). You wont’ have to worry about the current state of the operating system, setting up registry keys, creating databases, or anything else.

One step back from this would be to replace step one with “Run the dev machine setup script”. This script would install the VCS client, take care of any other hard dependencies and get you ready to go.