If you’re not currently saving your old builds, you should start now. Saving old builds can be very useful for a few reasons. First, it makes it quicker to start debugging problems that only shows up sin an old build. Second, it is insurance against the possibility of not being able to recreate your build environment correctly.
For hosted environments, where you control the deployment of the software, just keeping a couple of old builds around is probably enough. Being able to roll back if something goes horribly wrong can be a life-saver. For installed software, it’s important to keep all old builds that are still available in the wild or that customers might legitimately want for some reason. Practically, this works out to be all builds that you’ve released in some form to customers. During development, it can also be useful to have a few recent builds available that have only been released to the QA team to easily compare versions.
As a simple first step, you can simply store old builds on a file share. It’s not too much more work to store them in a version control system. Make sure to use a VCS that can handle large binary files easily - SVN is a good choice currently - so that you don’t have to worry about performance issues in the future.
Finally, when storing old builds, make sure to keep each configuration of the build. When a customer reports a problem in an old release, having a debug build of the code with symbols can rapidly decrease the time it takes to respond to their problem.