Monitoring: Corollary to "Release Early and Often"

Each day I get off the elevator and walk to my desk at Red Hat I am greeted by a very large sign that says "Release early, release often." This sign encourages us to get incremental results and get feedback on the software that we are working on early rather than uncovering a fatal flaw in design or implementation late in the process.

At DevNation 2014 I had the privilege of being a panelist for the Birds-of-a-Feather session "Best Practices for System and Application Monitoring" with Mike McGrath, Jordan Sissel, Langdon White, and Gene Kim. Gene Kim posted notes from the session that included the following tweet from Leslie Hawthorn (@lhawthorn on twitter):

Things need to be done earlier & be part of your culture. When your hair is on fire is not the best time to learn the new things.

The above tweet sums up the performance monitoring corollary to "Release Early and Often." So many professions practice and drill using the tools of their trade on a regular basis before needing them in an emergency. Why should it be any different for IT professionals and performance monitoring tools on computer systems? There are going to be unusual performance issues that come up, but we should avoid installing a new set of tools on the system and reading the documentation for the first time when the performance emergency is in progress.

We should use the performance monitoring tools during the less stressful times to understand the behavior of our important applications. Use the tools to identify likely bottleneck and performance limits before they become problems.