On Reducing Operational Chaos: the Costs of Toil

For anyone who has ever worked in production operations support, you know how your job can vary widely between a stable planned out day to the utter chaos of trying to hold things together. Ask yourself: do more days than not tend to feel like chaos? If so it probably leads to a feeling of helplessness in your daily work of not being able to make progress in what you do.

Its time to take control of the chaos, determine what is causing it, and work to eliminate the toil.

Where does the Chaos Come From?

---

Being in the thick of things will make it hard to see where the real problems lie. Here are some things to try which might help you to see trends in all the tasks you do each day to keep things running:

• Journal your daily tasks. Even if its just a simple log of the high level things you have done, that is good enough. At the end of your work week, sit down and look for some themes in the tasks.

• Look through your bug / time tracker. What were the common tasks you worked on each day? Can you discern a theme of different pieces of software or processes common to many?

• Track your time. If you are not sure as its not well documented what it is you are working on, once an hour set an alarm and write down what was done in the last hour.

Some Solutions

---

With a better understanding of what's going on, what can be done with it to minimize the feeling? Ask these questions of the list you generated above:

• Can it be automated? Is there some programming which can be done so this is no longer needed?

• Can it be stabilized? Are there deficiencies in how it was setup which can be re-architected or new a new technology that might allow for fewer problems?

• How can we balance the load? Is one part of the process overwhelmed and needs assistance to remain stable? These are usually infrastructure or database related.

• How can we re-architect for fewer issues? Sometimes its time to throw it away and start over leveraging better processes and technology.

• Communicate this list and the time its taking to management. Simply presenting the impact and cost of these items to your peers should motivate them to assist you in eliminating these tasks.

Wisely Spend Your Time

---

Looking back at the end of the day: were all of those things which you had to do that caused the chaos that important? What are you main goals for this quarter or year? Did they align with those goals? Is it your job to be doing these things? Is the task you are working on related to what you have planned for the week?

Make sure to focus your effort on items which are the most benefit to the company:

• Stability and reliability of your product.

• Improvements of your product.

• New ideas / technologies and products.

• Better support and service for your customers.