Patches and upgrades. How many outages – publicized and not – have resulted from insufficiently tested, ill-timed or skills-challenged software upgrade efforts?
Upgrades are a way of IT life. Successfully executed, however, results generally bring greater functionality, performance, ease of use or other benefits.
So it was when Peak 10 prepared to upgrade our managed back-up solution powered by CommVault to the most current version. This was no “Patch Tuesday” task. Since the upgrade would cover 3,300 installed clients processing more than 5,000 jobs daily, we took our time to understand how our large infrastructure would best accept that upgrade with minimal impact to customer operations.
Working in collaboration with the engineering groups in each of our data center markets to uncover and account for all possible customer issues, our cloud infrastructure team laid out a plan. We executed that plan first in our lab, running rollout simulations to look for potential issues and determine maintenance windows. We operate three CommVault environments within our infrastructure, so each was a bit different. From smallest to largest, each environment was upgraded successfully over successive days, the largest being 2,000 clients. Now that it’s done and the foundation is set, the work continues with documenting the process in detail and upgrading customers individually.
As routine as most operations such as this are at Peak 10 (this wasn’t one of those), we never treat them as routine. That would be a disservice to the customers who entrust us to have their backs and to do things right the first time. All the appropriate skills are brought to bear, all necessary parties are involved, and nothing is left to chance.
When we perform upgrades, bring out new products, evaluate new technology solutions, or just do our daily jobs, we do it for our customers’ benefit … to bring the best products available up to peak performance in order to keep customers’ data, applications and infrastructure available and protected all the time. This was a monumental task that was performed by the ability for a large team to synchronize and work in a unison that not only an engineer can see the beauty in, but our customers and prospects can appreciate as well. We don’t say, “We’re in IT Together” for fun, it has multiple levels of meanings and this example shows a few layers in itself.