Rapid7 Blog

IT Ops  

3 Core Responsibilities for the Modern IT Operations Manager

In the good old days, IT operations managers were responsible for maintaining the infrastructure, meeting service levels agreements, sticking to budget, and keeping employees happy. Life was not easy, but at least it was familiar. You knew your hardware, your software, your employees. You determined…

In the good old days, IT operations managers were responsible for maintaining the infrastructure, meeting service levels agreements, sticking to budget, and keeping employees happy. Life was not easy, but at least it was familiar. You knew your hardware, your software, your employees. You determined services levels based on what you could actually see and touch. You told people what to do and they did it. While IT was perceived to be an expensive cost center, it wasn’t an issue as long as the phones worked and the systems ran according to expectation. That was then, and this is now. Today IT operations managers can still see a lot, but they can touch very little. While some companies still own a significant amount of hardware, a diminishing number are making new hardware purchases. They’re moving toward Infrastructure as a Service (IaaS) solutions. IaaS makes hardware, storage, network, software, and all the work required to support such things opaque to the consumer, very much in the same way that the power generators at Niagara Falls become opaque to those using electricity. All the consumer sees are the power outlets in the wall and the monthly bill sent by the power company. By the same token, IT operations managers only see control panels and dashboards. The days of going over to the data center to inspect capital assets are gone. Now, procurement is about getting the best hourly rate for a given service. It’s a different way of doing business. Not only is the nature of hardware in the enterprise changing—software is too. Given the proliferation of using continuous integration and continuous delivery techniques to deploy a greater number of smaller, container-centric applications and services, today’s IT operations require less human interaction to make software available to end users. Most of the work is done by scripts and in some cases, scripts made by other scripts. Applications are becoming more fine-grain. The monolithic application is giving way to those that are composed of collections of microservices deployed as ephemeral containers configured and controlled via orchestration. Increasing the number of fine-grain deployment units and the rate at which they change decreases the likelihood that one person will know every detail of an application. As applications and services become smaller, the technical authority and responsibility associated with them becomes more decentralized. Whereas in the past a “department” owned an application, in today’s world of containers and microservices, small, autonomous teams are the responsible entities. The role of the IT operations manager is no longer that of the expert gatekeeper, but rather that of the knowledgeable facilitator. So then the question becomes, what are the essential responsibilities of the manager in modern IT operations? There are 3 essential responsibilities: To set realistic budgetary expectations To ensure and maintain IT operations transparency To mitigate the impact of automation on the workforce Allow me to elaborate. Set realistic budgetary expectations One of the benefits of implementing cloud-based IaaS and container-based architectures deployed by way of automation is considerable cost saving. According to David Bray, CIO at the Federal Communication Commission: “Back in late 2013, we were spending 85 percent of our budget just to maintain 10-year old legacy systems. However, after a dramatic shift to public cloud and commercial service provider, we saw our maintenance spend drop down to less than 50 percent of our budget.” That’s right, 50% savings. This is no small amount. It becomes the stuff of legends. It also becomes the standard by which expectations will be set if left unchecked. It’s only natural for stakeholders to think, “We had 50% savings this year. Getting another 50% savings next year is a fair expectation.” Yes, automation will reduce costs dramatically at first, but then over time the saving will flatten out. Big efficiencies and cost savings are typically achieved early on. It’s a typical pattern when moving to cloud-based infrastructures. Yet, many in upper management are accustomed to making an up-front investment in an initiative with the goal of long-term savings. Thus, those in upper management who are new to cloud-based transformations might treat the significant saving incurred initially as an indicator of good things to come, that greater saving are on the way. Yet such an expectation is inferred and unrealistic. Thus, the wise IT operations manager will do well to keep budgetary expectation fact based. The wise IT operations manager will ensure that the immediate and projected costs and benefits of his or her organization's activities are clearly stated and available to appropriate parties easily, on demand. In other words, transparency is everything. So then, how is transparency achieved? Read on. Ensure and maintain IT operations transparency One of the benefits of IaaS is the rise in reporting technology. Gone are the days when it took a day’s worth of labor at the end of the month to create and distribute budgetary information through the management chain. In fact, IaaS companies such as AWS, Microsoft Azure and Google Compute Engine make the latest usage and billing information available to consumers at the click of a button. Today stakeholders can be in the know all the time and should be. As a result, stakeholders throughout the enterprise want to be in charge of their information infrastructure. To quote Laurence Chertoff independent consultant to nonprofits and former Director of IT at Npower Inc.: “If the end user cannot modify or administer the technology quickly, I don't want it.” It doesn’t matter whether a technology is out on AWS or in a server room on the 3rd floor, people want access to their technology. Thus, instead of being a powerbroker of information dissemination, the role of the modern IT operations manager is to make sure that people can get at the information they need, when they need it, in a manner that is easy, appropriate and secure. In addition providing the transparency that allows users to do as much for themselves as they can and want to do, items such as incident tracking, operational costs, service issues, and project workflow data need to be apparent and accurate. At the end of the day, the modern IT operations manager is a facilitator, not a roadblock, to information transparency. Making systems flow with as little effort as possible is one of the the essential responsibilities of modern IT operations management. Mitigate the impact of automation on the workforce Automation has an impact. Always has, always will, ever since the days when Dutch windmills ground wheat into flour faster than a human ever could. The good news is that automation technology made it possible for many more people to have more bread at a cheaper price. The bad news is that people who manually ground wheat into flour had to find other types of work to do in order to survive. The same holds true in IT operations. For better or worse, increasing the use of automation in IT operations means that the way people work has changed and will continue to do so. Steve Mays, CTO of Trizic has an interesting take on the matter: “Where we used to depend a LOT on the ‘art’ of super talented people who had a widely varied background from operations to development to help us build and manage systems and software, we now have automation in the cloud. In the past, [engineers] hand crafted the product to be artisanal and got a lot of satisfaction from a job finely crafted. [Today] we look at IT operations more like an assembly line. My mission is to turn out 100K units vs. beautifully [releasing] only 10. From a product output and quality perspective, this is a good thing. However, the systems are no longer your ‘friends’ that you have a ‘close personal relationship to’.” IT operations will become more automated, not less. The use of human labor will fall into two categories. One category is that work which requires creativity and imagination, for example system design and incident troubleshooting. The other category is work which is predictable and repetitive—requirements gathering and capacity planning. Eventually all predictable and repetitive work will be automated. It’s the nature of the dynamic. If machine intelligence can observe a pattern for a long enough period of time, eventually the machine intelligence can emulate the pattern—think speak recognition. Thus, the modern IT operations manager will be subject to a pressure that is new to the IT operations—how to plan for the continuing obsolescence of a portion of the workforce. To date, IT operations managers worrying about employee obsolescence has been minimal. The conventional thinking is that IT operations employees have always been in demand. If a job is lost in one company, a new job can be found in another. But, as comprehensive automation becomes more pervasive, the notion that there will always be another job might be a faulty assumption. Of course, the IT operations manager can simply not care. The problem is you can’t fake sincerity for a long period of time. As more automation sets in and employees face increasing pressures to exert more creativity or work with new, more difficult technologies in order to stay viable, many will turn to management for help. If that help appears as a lot of talk with no action, eventually employees will figure it out. The result is a workplace in which morale low and personal investment is nonexistent. In such an environment the quality of service degrades and eventually costs increase. Not caring is a solution with a very shallow horizon and poor ROI. The alternative is to take a new approach. Modern IT operations management is about taking the time to understand the impact of automation upon the workforce and having the foresight to take realistic steps to address the impacts early on. At the least, the impact of a given automation event needs to investigated and articulated in a transparent manner, particularly with regard to the human impact. Then, realistic strategies to address the expected impact need to be devised. If such strategies require accepting the substitution of human labor with automation, even if the substitution means the cost of an employee’s job, the enlightened IT operations manager will promote such discussions and foster ways to find solutions for the problems at hand. In the old days, the IT operations manager would just pass the issue of impact of automation on the IT operations workforce onto HR. However, given the decentralization of both authority and expertise brought about by automation itself, HR is no longer the expert in these matters. HR simply does not know enough. The IT operations manager is now the authority and expert, even in matters of personnel. Mitigating the impact of automation on human employment will be a growing concern of managers in IT operations. The issue is new and daunting. Yet, for some managers it will be an exciting opportunity. Those that can succeed in finding ways to mitigate the impact of automation on those they manager will stand at the forefront of the profession. Putting It All Together In the world of IT operations, the old days of manage by edict and command are over. Steve Mays from Trizic sums it up well: “Top down is 100% impossible now. My 30+ year background in tech isn’t that useful anymore. I used to guide folks with my experience. Now I guide them to exercise restraint, build good enough features but with high levels of testing to ensure fewer issues. I let them tell me all about the new frameworks/libraries/tech that they think we should use.” Today, given the continuous demands that enterprises make upon IT operations, IT operations managers need to provide to their stakeholders more complex services, at faster rates of implementation and higher levels of reliability. Automation technologies combined with greater transparency into IT operations are key elements for meeting these demands. Also, given the growing trend to decentralization, IT managers can no longer be expected to be know-it-alls. Real knowledge and authority resides in the small teams providing the solutions needed. The job of the modern IT operations manager is to be a knowledgeable facilitator, one who establishes and maintains an appropriately open and transparent environment from which realistic performance and budgetary expectations can be established. The modern IT operations manager sets the guidelines in which good work can happen. Also, given the growing prevalence of advanced automation technologies on the IT landscape and the impact that such automation will have on human employment, the modern IT operations manager will at the least, articulate the human ramifications of automation initiatives at hand and foster an environment in which reasonable discussion about the impact of automation can take place. The modern IT operations manager understands the dynamics of technical authority within the decentralized enterprise and the importance of providing timely, accurate information to support such authority. He or she knows how to both inspire and guide others to find the tools and techniques necessary to allow the enterprise to compete successfully in the marketplace for the benefit of all it touches—customers, employees, and shareholders. The modern IT operations manager understands that the real power of his or her position is to empower others to act reasonably, affordably, and safely. This is no longer power expressed by edict. Rather it is power that comes from influence. Power expressed as edict has the shelf life of a manager’s tenure. Power that comes from influence lives on well after the manager has left the company. I’ll leave it to the reader to decide which type of manager he or she wishes to be. For information on Rapid7’s IT operations solutions, click here.

Introducing InsightOps: A New Approach to IT Monitoring and Troubleshooting

Today we are announcing the general availability of a brand new solution: Rapid7 InsightOps. This latest addition to the Insight platform continues our mission to transform data into answers, giving you the confidence and control to act quickly. InsightOps is Rapid7's first IT-specific solution, enabling…

Today we are announcing the general availability of a brand new solution: Rapid7 InsightOps. This latest addition to the Insight platform continues our mission to transform data into answers, giving you the confidence and control to act quickly. InsightOps is Rapid7's first IT-specific solution, enabling users to centralize data from infrastructure, assets and applications, so they can monitor and troubleshoot operational issues. Getting in with the IT crowd Every day, IT and security teams work hand-in-hand towards keeping their organizations secure, optimized and operational. Yet today's IT environment is more complex than ever. Infrastructure is hosted across physical servers, virtual machines, Docker containers and cloud services. The corporate network is accessed by internal and remote employees, from a mix of known and unknown devices that are all using applications, both internally hosted and cloud-based. This complexity creates enormous amounts of data, dispersed across the modern IT environment. Managing this data is critical, but for most resource constrained IT and security teams, it's simply too complex or too expensive to monitor it all. And unmonitored IT data creates risk. That's where Rapid7 comes in. Today, our customers leverage the Rapid7 Insight platform to collect data from across their entire IT environment for identifying security vulnerabilities with Rapid7 InsightVM and catching attackers in the act with Rapid7 InsightIDR. InsightOps builds on this, enabling them to manage and optimize IT operations across their technology landscape. Introducing Rapid7 InsightOps We built InsightOps to be easy to set up and scale. It requires no infrastructure to run, no configuration of indexers to search, and you can collect data in any format from anywhere in your environment. With your data centralized in one place, it's easier to then monitor for known issues or anomalous trends. Monitoring with InsightOps helps you proactively address issues before they become widespread. Ultimately, InsightOps was built for turning IT data into answers. With features like Visual Search and Endpoint Interrogator, it's easier to get answers from your data without ever even having to type a search query. And log data is just the beginning. Sometimes you need answers directly from your IT assets, like what software is running on an employee workstation or which servers are over 75% disk utilization. InsightOps combines log management with IT asset visibility and interrogation, enabling you to trace issues all the way from discovery to resolution. Ready to transform your unmonitored IT data into answers? Start your free 30-day trial of InsightOps today.

Announcing InsightOps - Pioneering Endpoint Visibility and Log Analytics

Our mission at Rapid7 is to solve complex security and IT challenges with simple, innovative solutions. Late last year Logentries joined the Rapid7 family to help to drive this mission. The Logentries technology itself had been designed to reveal the power of log data to…

Our mission at Rapid7 is to solve complex security and IT challenges with simple, innovative solutions. Late last year Logentries joined the Rapid7 family to help to drive this mission. The Logentries technology itself had been designed to reveal the power of log data to the world and had built a community of 50,000 users on the foundations of our real time, easy to use yet powerful log management and analytics engine. Today we are excited to announce InsightOps, the next generation of Logentries. InsightOps builds on the fundamental premise that in a world where systems are increasingly distributed, cloud-based and made up of connected/smart devices, log and machine data is inherently valuable to understand what is going on, be that from a performance perspective, troubleshooting customer issues or when investigating security threats. However, InsightOps also builds on a second fundamental premise, which is that log data is very often an incomplete view of your system, and while log and machine data is invaluable for troubleshooting, investigations and monitoring, it is generally at its most powerful when used in conjunction with other data sources. If you think about it, knowing exactly what to log up front to give you 100% code or system coverage is like trying to predict the future. Thus when problems arise or investigations are underway, you may not have the complete picture you need to identify the true root cause. To solve this problem InsightOps allows users to ask questions of specific endpoints in your environment. The endpoints return answers to these questions, in seconds, in the form of log events such that they can be correlated with your existing log data. I think of it as being able to generate 'synthetic logs' on the fly - logs designed to answer your questions as you investigate or need vital missing information. How often have you said during troubleshooting or an investigation "I wish I had logged that…”? Now you can ask questions in real time to fill in the missing details e.g. “who was the last person to have logged into this machine?” InsightOps combines both log data and endpoint information such that users can get a more complete understanding of their infrastructure and applications through a single solution. InsightOps will now deliver this IT data in one place and thus avoids the need for IT professionals to jump between several, disparate tools in order to get a more complete picture of their systems. By the way - this is the top pain point IT professionals have reported across lots and lots of conversations that we have had, and that we continue to have, with our large community of users. To say I am excited about this is an understatement - I've been building and researching log analytics solutions for more than 10 years and I truly believe the power provided by combining logs and endpoints will be a serious game changer for anybody who utilizes log data as part of their day to day responsibilities -- be that for asset management, infrastructure monitoring, maintaining compliance or simply achieving greater visibility, awareness and control over your IT environment. InsightOps will also be providing some awesome new capabilities beyond our new endpoint technology, including: Visual Search: Visual search is an exciting new way of searching and analyzing trends in your log data by interacting with auto-generated graphs. InsightOps will automatically identify key trends in your logs and will visualize these when in visual search mode. You can interact with these to filter your logs allowing you to search and look for trends in your log data without having to write a single search query. New Dashboards and Reporting: We have enhanced our dashboard technology making it easier to configure dashboards as well as providing a new, slicker look and feel. Dashboards can also be exported to our report manager where you can store and schedule reports, which can be used to provide a view of important trends e.g. reporting to management or for compliance reporting purposes. Data Enrichment: Providing additional context and structuring log data can be invaluable for easier analysis and ultimately to drive more value from your log and machine data. InsightOps enhances your logs by enriching them in 2 ways, (1) by combining endpoint data with your traditional logs to provide additional context and (2) by normalization your logs into a common JSON structure such that it is easier for users to work with, run queries against, build dashboards etc. As always check it out and let us know what you think - we are super excited to lead the way into the next generation of log analytics technologies. You can apply for access to the InsightOps beta program here: https://www.rapid7.com/products/insightops/beta-request

Featured Research

National Exposure Index 2017

The National Exposure Index is an exploration of data derived from Project Sonar, Rapid7's security research project that gains insights into global exposure to common vulnerabilities through internet-wide surveys.

Learn More

Toolkit

Make Your SIEM Project a Success with Rapid7

In this toolkit, get access to Gartner's report “Overcoming Common Causes for SIEM Solution Deployment Failures,” which details why organizations are struggling to unify their data and find answers from it. Also get the Rapid7 companion guide with helpful recommendations on approaching your SIEM needs.

Download Now

Podcast

Security Nation

Security Nation is a podcast dedicated to covering all things infosec – from what's making headlines to practical tips for organizations looking to improve their own security programs. Host Kyle Flaherty has been knee–deep in the security sector for nearly two decades. At Rapid7 he leads a solutions-focused team with the mission of helping security professionals do their jobs.

Listen Now