Key Ingredients for Run Book Automation Systems

Modern business and utility computing incorporates numerous layers of technology, which by itself creates several discrete pockets of and serviceable levels for automation. In terms of IT operations oversight, there is no individual component, nor combination of components perhaps more important than a coherent and holistic perspective on data center management. This is particularly true when it comes to process automation and incident handling. Centralized management with service provisioning is both central to and instrumental for the next generation of enterprise computing management development.

Achieving an organic self-sustaining IT infrastructure is possible, but it seldom comes easily, cheaply or instantly. The trick to designing an appropriate all-purpose IT management solution comes from developing the technology that assists enterprise and government organizations in moving beyond tools restricted to disparate and disjoint IT domains. This enables such outfits to work across isolated IT infrastructure silos, and to improve day-to-day operational efficiencies across the board. By the same token, well-orchestrated run book procedures also eliminate more error-prone, labor-intensive processes and can deliver services both more efficiently and costeffectively—starting at the ground level and working upward from there.

Breakthrough advances in RBA/ITPA and data center automation products demonstrate a strongly cohesive coupling between enterprise management orchestration technology and best practices for managing business processes and workflows. In fact, RBA has proven itself to be highly effective in reducing manual labor, lowering operational costs, and improving overall operational efficiency. Wide-scale enterprise automation was once an empty promise, a vaporware solution that went a long time without any deliverable goods to back it up. Today, there is a burgeoning throng of vendor candidates ready to fill any voids in this once empty market space. Some might take this as proof-positive that nature—in this context, the nature of business—abhors a vacuum; we believe it simply means that tools and technologies have advanced to the point where practice can deliver on promise, if aided and abetted by appropriate tools and technologies.

Think of workflows as automated sequences of steps and transitions that perform programmed operations and return execution results. Every workflow sequence contains an entry (start) and end (stop) point. Each step performs some given operation, which may or may not have inputs, and may or may not produce outputs (though some kind of status or success report is invariably created at each step along the way). A step's operation can itself be a workflow, a mere subset of a workflow, or a subflow within the same scope. This approach enables implementers to created nested and reusable workflows wherever and whenever they're needed.

In the first chapter, we set the stage for defining, identifying and creating workflows with a focus on prioritizing run book procedures that organize, orchestrate, and bind them together. Here, we construct a functional model that incorporates the key ingredients, integral concepts, and essential elements that make up an RBA system. We also introduce some specific citations and examples from existing RBA or ITPA vendors already active in this exciting new market space.

Managing the Unmanageable

As stated in Chapter 1, unplanned downtime and routine "firefighting" tasks consume more than half of IT's human and financial resources, just to support ongoing production and routine, recurring activity. Alas, this leaves little left over for planning, purchase, or future development, and often serves to stifle than foster innovative growth. Mostly manual processes and entirely adhoc "bubblegum and baling wire" solutions for change management may take weeks to implement when undertaking end-to-end server and network provisioning. Over 60% of operational funds and time are wasted on basic, repeatable maintenance tasks that incur further performance and economic penalties in the long-term.

Furthermore, there remains a lack of centralized visibility or holistic visualization capability and control over the dozens of site-specific, task-oriented "tribal knowledge" scripts that are often at work in so many organizations. The end result is a disjointed management effort that operates from an incomplete and untutored perspective, and that fails to take future needs and development into account. At worst, this guarantees certain failure. At best, this vicious cycle creates resource churn where too much effort gets wasted on "same thing, different day" and not enough effort goes into creating sound, replicable processes and procedures for handling routine efforts, while also designing and deploying tools and technologies carefully chosen to improve productivity and enhance operating efficiency.

Change management is an IT Service Management (ITSM) discipline whose objective its to ensure the application of standardized methods and procedures for timely and efficient handling of any changes to a well-controlled and –documented IT infrastructure. This minimizes the frequency and impact of related incidents upon services that stem from reactive or ad hoc changes (perhaps arising from new or revised legislative mandates or requirements) or proactive changes (possibly, internal improvement initiatives, or efforts to extend existing or add new functionality to the IT infrastructure).

ITSM is a discipline for managing IT systems according to the business perspective on IT's contribution and value to the bottom line. ITSM is platform-neutral, process-oriented, and unconcerned with specific details of how to operate a particular vendor's product or their technical implications for underlying managed systems. In this sense, ITSM shares common interests with process improvement frameworks and methodologies such as Business Process Management or Business Services Management (BSM).

Managing Change Within IT Infrastructure

Change management systematically approaches change from both organizational and personal levels. The change management process includes three separate aspects that include the following responses to change: efficient and prompt handling of changes, minimization of change impact on service quality, and ongoing, incremental improvements in day-to-day operations and service quality. A proactive approach to dealing with change lies at the core of these three aspects of the change management process. Organizationally, this means defining and deploying procedures and technologies to deal with planned (and unplanned) changes within a business environment. Ideally, this results in the business profiting from changing opportunities.

The change management process is responsible for the following:

IT infrastructure hardware (enterprise-wide)
Communications equipment, services, and applications
System software and software components
All related documentation and procedures

Successful change management adaptation requires getting used to encountering forms of change that are well beyond anyone's individual control. A thriving business is a perpetually adaptive one, and might benefit from establishing a structured methodology for responding to changes in the business environment (for example: economic fluctuations, competitor threats, or resource availability issues). Or it might require development of coping mechanisms for responding to inevitable changes in the workplace (such as changes to federal regulations, state policies, imposition of mandatory methods or technologies, or other changes to the workplace climate and environment).

The IT Infrastructure Management Problem Revisited

Change orchestration and incident management seek to avoid inefficient handling or processes whenever possible, and to mitigate such issues as change may cause from time to time. High alert volumes and a complete lack of process consistency make provisioning automated tasks difficult if not nearly impossible, so one goal in the change management process must be to minimize alerts and to enforce consistency as much as possible. One thing is inarguable: whenever tool integration is non-existent or weak across an enterprise, incident response personnel are often unable to resolve the majority of alerts. An ideal RBA platform should therefore dramatically increase productivity levels, drastically reduce development support time, and proffer significant savings in annual labor costs, simply because it imposes order on chaos, enables alert resolution, and keeps things working coherently and consistently over time.

In the same vein, now consider disaster recovery and automated maintenance procedures. These all-too-manual processes also incur time, cost, and resource penalties where support personnel must handle tens or perhaps even hundreds of necessary workflows manually, catch as catch can. These workflows too can be adequately defined and easily automated within a proper RBA solution. Reduced Mean-Time to Recovery (MTTR)–particularly for disaster recovery—and substantial workflow-related cost savings are two significant elements in the collection of RBA benefits.

Remember that Mean-Time to Recovery (MTTR) measures the time between service interruption and subsequent service restoration, including everything from problem diagnosis to problem repair. When an unexpected or unanticipated change occurs in an uncontrolled, unmanaged fashion (or within such an environment) MTTR is dominated by problem diagnosis. The longer diagnosis takes, in other words, the long users must wait before "business as usual" is restored.

Typically, upwards of 80% of MTTR time goes into the diagnostic phase, except where changes are properly documented and appropriately managed. Consequently, shorter diagnosis periods translate directly to shorter MTTR periods. Critical time can easily be lost in a flurry of phone calls or pages, impromptu meetings or strategy huddles, and possible fixes and applications. Also consider time lost to preparing and readying a solution that proves false or a total failure. RBA short-circuits all of these things, and helps take incident staff from alert to resolution with a minimum of muss, fuss, and manual effort.

Also remember that 80% of downtime is a result of human error. Manual management processes inherently introduce delays into service delivery and carry the risk that human error will delay service deliveries still further. Avoiding these two problem areas can help to bolster performance in a variety of measurable attributes, while also significantly reducing the time it takes to work through the problem resolution process.

MTTR ranges from self-resetting devices that may require seconds to fully cycle up to entire systems that can take days to replace or restore to full operational capacity. Systems or components may have a MTTR of infinity—indicating no redundant or immediately serviceable or replaceable parts, which means there is nothing to take over control in the event of an inoperable condition or terminal failure.

This increases another measurable value: mean down time (MDT). In organizational management, MDT is the average period a system remains non-operational including all time associated with repair, corrective and preventive maintenance, self-imposed downtime and any logistics or administrative delays. Such circumstances indicate longer recovery periods that likely involve on-call pages, emergency service requests and a scramble for immediate replacement parts or platforms.

The IT Management Solution

An IT run book facilitates a correct working order for all regular housekeeping chores, maintenance tasks, and general data center-related activities. Experience shows that this process is a crucial, time-consuming undertaking fraught with potential for human error to occur. Convert the contents of your IT run book into a set of automated procedures and your IT infrastructure can operate 24 hours, seven days a week at optimal efficiency and with minimal operator intervention.

Let's recap some typical IT-related problems as they map into RBA system features:

Automate IT operations processes—Intuitive, visual drag-and-drop scripting tools facilitate rapid creation and contain readily-deployable process definitions to run administrative application and general data center and workloads. RBA lets you embed operational logic and exception handling conditions into automated processes, and eliminate costly missteps resulting from error-prone yet essential administrative tasks. Not only does this speed routine administration, configuration, and provisioning tasks, it also helps enterprises to avoid unscheduled downtime.
Orchestrate data center workload—RBA lets you dynamically manage dispatch and execution of run book procedures, automate responses to operator requests, and proactively monitor the entire enterprise landscape. RBA also enables IT to synchronize parallel processing of database backups, harmonize workloads across multidisciplinary platforms, and unify business application handling to reduce scheduled downtime.
Integrate with IT infrastructure tools—RBA permits IT operations to communicate and combine with existing systems management tools using proven interfaces and automate raising of alerts, events, and trouble tickets whenever jobs, tasks or procedures fail. RBA can also integrate handling of server, database, and network monitoring thresholds and triggers with other operational tools and procedures. Even when incidents occur and must be handled, RBA can help to speed that process using automation to facilitate processes, provide notification, and to drive human action (where necessary).
Ready-made, readily-deployable templates—RBA helps to accelerate implementation and deployment using predefined templates to map common IT operational procedures within a data center to well-documented and –understood automation sequences and workflows. Most RBA systems enable IT to craft and customize repeatable in-house templates, or to work from a library of well-defined workflows created in advance. Either way, such templates permit IT to interact with modular design tools and generally, to simplify their operations within a typically complex IT landscape.
Centralize management and monitoring—RBA systems come equipped to let IT monitor and manage workloads and workflows from a centralized point of control. This enables IT to use RBA to track progress, handle exceptions, and review outcomes using an enterprise-grade dashboard. (Some RBA systems even provide real-time graphical perspectives into all areas of processing activity on enterprise systems and networks.)

Let's take a deeper look into some of what the market currently offers. Even though there are still a lot of growing pains for both vendors and customers to endure, nor a lot of standardized concepts or components around, there are plenty of worthy RBA solutions available for purchase and use today.

Enterprise-Grade IT Management Solutions

First and foremost, an RBA solution must be vendor-agnostic. It should differentiate between hardware architecture and software platform only in how specific processes or tasks are handled separately in each case. Creating an RBA process across the enterprise should be so completely vendor-neutral as to make no requirement for advanced or specialized knowledge of the enterprise platform portfolio. It should simply model, automate, measure and monitor workflows and workloads in a seamless, well-integrated fashion.

Through a strategic combination of task, process and decision automation, junior support personnel can reduce time and energy spent escalating issues for senior support personnel to diagnose and address. Expanded monitoring capabilities can anticipate common data center issues so that IT can proactively support business processes, instead of reacting or responding after-the-fact.

Available Market-Ready RBA Solutions

Several big name vendors, lesser-known acquisitions by big name vendors, and smaller companies looking to make names for themselves have all thrown their hats into the collective ring that is RBA/ITPA. There is no shortage of key players manning this open playing field, a space populated by both big names looking to increase their portfolios, and newcomers seeking to make a big splash. This medley of providers offers a rich collection of industry-focused solutions along with a diverse range of industry-specific capabilities, some of which are interrelated and inter-dependent.

Unofficially speaking, RBA vendors divide into two camps: generic process and specific process providers. Generic providers (which include companies like RealOps/BMC, Opalis, and iConclude/HP) aim for holistic computer system automation. Specific RBA providers (such as LanDesk, Enigmatic, BladeLogic, and Opsware) offer automation for various tasks. For example, while both BladeLogic and Opsware offer IT orchestration for server provisioning, they do not rely on their own technology to handle the provisioning aspect of that task.

To date, Optinuity has launched a re-branded flagship product called Oasis, an autonomic policy management system. Opalis maintains its RBA footing with the presence and promotion of its Opalis Integration Server, which boasts Service-Oriented Architecture (SOA) support and rapid integration capability. Big name vendor HP also markets a current (re-branded) offering called HP Operations Orchestration, built upon what was formerly iConclude's OpsForce platform, for critical management software integration and enterprise-grade standard operations process automation. BMC acquired existing outfit RealOps to extend its reach and assert its role in business service management and IT process automation leadership. At the same time, NetIQ introduces its Aegis platform to control and automate IT processes, while Stratavia unveils an industrial-strength data center automation platform called Data Palette. Choices in this space abound, where savvy buyers look for stable offerings, strong synergy with existing investments, and "good deals" on up-front and ongoing costs involved in acquiring and deploying RBA solutions.

Elements of the RBA Environment

RBA is an enterprise-grade administrative adhesive that binds together existing trouble ticketing, asset management, source control, system monitoring and other IT operations software from various competing vendors. Certain basics components in the RBA environment remain instrumental to its success within the enterprise. Thus, we must identify such necessary properties and frame your expectations according to how you view and value your IT infrastructure and support personnel, to help you select and acquire the right solution to meet your needs and your situation.

Essential Traits of an RBA Solution

At its most basic, RBA should provide four crucial and essential elements of enterprise management, including:

Resolve incidents—Automate common, repeated processes for alert and incident diagnosis and resolution. Eliminate alert floods, empower frontline operations, and reduce escalation volume.
Change orchestration—Uniformly create, apply, and manage change across all tiers of the IT infrastructure. Enforce process compliance, automatically generate audit trails, and automate changes end-to-end.
Automate maintenance—Automate routine maintenance and common repeat tasks. Improve IT personnel productivity, enhance IT resource availability, and free-up key development resources for strategic tasks.
Integrate tools—Create end-to-end automated processes across IT silos and isolated tools. Reinforce IT best practices, seamlessly integrate isolated silo tools and unify visibility and control.

Ideally, the RBA environment will require no tribal knowledge and little programming experience to operate and maintain. It will also encourage and enforce best practices, and combine with software-centric approaches to task automation. And a good RBA environment should come equipped with adequate and complete technologies to do these things pervasively across an enterprise environment that takes complete shape once all the core architectural underpinnings are in place.

Stated more clearly, a complete RBA solution should address all of the following business needs:

Efficient and effective incident response
Improved quality of service and service availability
Reduction of man-made errors and repeatable tasks
Decrease Mean-Time to Recovery (MTTR) metrics
Enhance Mean-Time to Failure (MTTF) measurements
Offer enterprise-wide change event orchestration
Assist, if not enforce, standards and regulatory compliance
Implement the IT Infrastructure Library (ITIL) ITPA framework of best practices

These first three points are solidly covered in any RBA process that automates enterprise-level tasks, and the next two points (MTTR and MTTF) appear as a beneficial byproduct of those three capabilities. Special will be needed for systems that process sensitive or personally identifiable information, which is governed by strict and specific regulations to ensure its proper care, handling, auditing, use, and distribution. Therefore, a proper RBA solution must also integrate and coordinate with existing infrastructure security elements such as role- or task-based access controls. Finally, ITIL framework and best practices compliance ensures proper alignment between business goals and the RBA solution.

An appropriate RBA solution also provides easy to use, ready-made workflows that are readily usable straight out of the box. In addition, existing shared and/or custom scripts or workflows can be simply imported and incorporated into the RBA task scheduler alongside pre-packaged template items. Much of the heavy lifting should already be done, created by way of a centralized design that incorporates functional roles and facilitative capacities to orchestrate an enterprise network environment.

Platform Neutrality

Traditional automation methods—primarily custom coding and scripts based on tribal knowledge for ad-hoc situations—may be great for handling simple, short-sighted tasks. However, they lack best practices, change management, relevant documentation, and essential flexibility required within any IT operations environment. Within that environment, business rules of engagement and desired configuration settings change frequently. For truly effective, highly agile automation an RBA platform should not conceal custom code behind its workflows, or within script code per se.

Therefore, a practical RBA solution should also be easy to customize to meet a customer's every need, and tweak to comply with an implementer's every want, all without requiring full-blown programming skills or knowledge. To all intents and purposes, a good RBA solution should itself be both a vector for implementing decisive administrative procedures, and a vehicle for executing series and sequences of simple, process-oriented tasks. No enterprise should find itself forced to rely on the tribal knowledge about IT infrastructure administration and management. Any such reliance must be entirely eliminated through a well-chosen, well-appointed, welldocumented, and well-used RBA solution (and use its workflows to get tribal knowledge out of its holders' heads, and into that system where it can be completely explored, explained, and understood by all).

Write Scripted Tasks

A key feature of any lifecycle-driven RBA automation platform is the ability to easily author, rapidly deploy, then simply run and report on a variety of process-oriented activities. You should be able to author scripts quickly, or create and implement workflows, using a drag-and-drop interface with out-of-the-box workflow templates and integration adapters. You must also be able to import existing scripts directly and easily, and utilize an RBA's built-in debugger for diagnostic walkthroughs on failed or problematic procedures, as well as new items under development.

Deploy Scripted Tasks

Another key aspect in a RBA automation platform is its ability to publish and deploy scripts, import or export shared workflows, and automatically generate documentation for procedural workflows. Ideally, such a system incorporates an enterprise security model (perhaps a rolebased access or RBAC control scheme) and provides single sign-on (SSO) integration of some kind (perhaps with Active Directory or Kerberos).

Single Sign-On (SSO) is a form of access control that enables one-time user authentication and grants access across an expanse of resources from multiple software systems. Alternatively, enterprise reduced sign-on is preferred by technically accurate types, since there isn't any real SSO within heterogeneous IT infrastructures.

Execute Scripted Tasks

In a well-built RBA automation environment, a visually guided execution mode—a logical counterpart to visual scripting elements—provides a guided tour through run book procedures before you switch over to fully-automated modes of operation. While this may not be a standard feature in all such packages, it is a welcome component in some RBA platforms. This mode may be scheduled and designed to operate with gated transitions, and may even allow authorized users to browse through or search scripts and workflows using some kind of built-in browser or dashboard interface. Especially when IT professionals are learning how to use an RBA system, the ability to explore what's already there, and to learn by doing more of what's already been done, can be absolutely invaluable.

Report Scripted Tasks

Whenever a workflow executes, it should conclude with a follow-up audit for later execution path or performance analysis. Audit records can describe start times, duration, initiating user credentials, actions performed, and the end results. Every such record can be vital in ensuring that certain systems handle specific types of information in compliance with relevant state and federal regulations, or in keeping with industry best practices and procedures. Finally, in the vast majority of RBA systems, each workflow instance or script concludes with details generated and recorded by the system itself, to provide the most complete and explicit documentation possible, including audit tracking and results information.

A quick run-down on typical IT troubleshooting and repair: it is an ad-hoc or largely improvised procedure where results may be noted on a ticket but are often not recorded for posterity or subsequent further analysis. This alone makes such an execution strategy impracticable for Sarbanes-Oxley (SOX) and Health Insurance Portability and Accounting Act (HIPAA) audit requirements. For best results, all RBA workflows must be documented, including data automatically generated by the RBA system itself, and automatically audited, to ensure compliance with applicable law and regulations, enterprise policy requirements, and best or prudent industry practices.

RBA Platform Scalability and Flexibility

Enterprise scalability and scope is essential for sustaining business workflow patterns and operational trends. Over time, these trends may shift to favor a different technology factor or facility, or adopt some new process or procedure. This can quickly and dynamically change the usability and usage benefits of a weak RBA solution very much to the negative.

To preserve its livelihood and maintain a steady stream of customers, an RBA development team must design a platform that is rigid enough to uphold current enterprise values and flexible enough to meet upcoming enterprise targets and accommodate ever-changing business processes and trends. Therefore, your RBA solution should be extensible in a number of critical ways— from how it handles new tweaks to old techniques, to how it incorporates new techniques arising from hitherto unknown or unforeseen technical challenges.

An ideal RBA platform is highly scalable so that it can adapt to any network conditions and system criteria. It must also be globally deployable and deliverable across diverse environments that include numerous different hardware architectures and software platforms. This same solution must also be secure in and of itself, and provide enhanced security features to integrate and incorporate existing security functions, foundations, and facilities. And your RBA platform should also be extensible, so that unforeseen connections, unexpected tasks, and brand-new workflows or scripts can be handled with appropriate dispatch.

Integration adapters are vital to develop an adaptable RBA for your organization or enterprise. Adapters provide plug-in points for technology and information silos to create a more cohesive enterprise management platform among a diverse set of circumstances, components, and connections. A well-designed RBA platform itself can integrate with third-party systems and security management products to better leverage an organization's existing assets. This also results in greater cohesion among multiple disciplines and vendor management tools.

An integration adapter resides at the lowest layer of the RBA totem pole. It supports an agentless means for interacting with infrastructure elements, provides easy mechanisms for extensibility, and seamless integration with custom scripts, workflows, or existing tools within otherwise self-contained or isolated silo environments.

IT infrastructure integration adapters proffer a vast array of connection points for interaction with various components or resources on the network and the systems it serves. Some of these components include system services: the command-line, Telnet and Secure Shell (SSH), Web services (HTTP, HTTPS), File Transfer Protocol and Secure FTP (SFTP). Other components utilize languages such as C#, Perl, VB Script, Java, and JavaScript.

Bidirectional capabilities allow RBA software components to operate in a collective, coordinated fashion. This means one component can meaningfully coexist and cooperate with other RBA components to achieve a single ITPA management and monitoring perspective. And virtually every component should be aware of the data it possesses as well as the processes it perpetuates.

Built-in role-based access controls (RBAC) will enable suitably-equipped RBA platforms to integrate easily within existing enterprise security models. In such cases, these systems will invariably include adapters that enable them to obtain and share security information, particularly where it relates to access controls and permissions, with Lightweight Directory Access Protocol (LDAP) or Active Directory (AD) based environments. This means that these RBA platforms can drop seamlessly onto enterprise networks that rely on LDAP or AD based directory services for security policy and access control, and will become instantly accessible to authorized IT administrators once the proper adapters are installed and interconnections forged and put to work.

Best Uses for RBA

Now that you are apprised of the essential elements that comprise the RBA platform, how do you use it? What are the best uses and best practices for a proper RBA setup? Understanding its operational aspects is one matter; utilizing its important applications and leveraging its integral properties advantageously is another.

The ITIL (Information Technology Infrastructure Library) framework is a standard approach to IT management and deployment originally developed in the Central Computer and Telecommunications Agency (CCTA) in the UK government. ITIL outlines best practices for ITrelated activities, and today drives the ISO/IEC 20000 standard for IT Service Management elements. ITIL defines numerous core areas of IT activity, including Service Strategy, Service Design, Service Transition, Service Operation, and Continual Service Improvement. Service Operation includes both Service Delivery and Service Support. Within the ITIL framework, service support activities that feature as daily operational IT tasks include managing the following: incidents, problems, configurations, changes, and software releases. As such, any well-built RBA solution should directly correspond to these ITIL processes so that IT operations can implement process automation out-of-the-box, and take advantage of the collective wisdom and best practices surrounding these activities so readily available within the ITIL Service Support mindset and documentation.

Automating Basic Maintenance and Upkeep

A fundamental precept of the RBA process is to rescue frontline "early responders" from becoming encumbered by unnecessary routine or repetition, and reduce the frequency and duration of escalation response periods. Automating basic maintenance and upkeep is crucial to maintaining a capable support team and IT infrastructure. Removal of manual processes and time-consuming hands-on tasks enables a much greater range of flexibility and mobility for the existing workforce without requiring additional resources or increased headcount.

Expect any good RBA solution to eliminate several key areas of performance bottlenecking, resource saturation, and personnel overburdening. The need for better IT efficiency and greater infrastructure agility fuels demand for this breed of optimization solution. Better yet, RBA enables more effective, and dynamic capacity optimization, and also helps align IT more closely with business goals and enterprise objectives.

Recurring software updates, platform patches, upgrades, hotfixes, bugfixes, roll-ups and service packs are all major sinks for wasteful manual efforts. When RBA is applied to automate such tasks, many enterprises discover that they can make much better use of IT, help desk, and technical support staff, and realize immediate returns on that technology investment. In far too many IT environments, managing these kinds of routine IT activities typically require manual intervention at some point, if not manual operation from start to finish. RBA software can define, deploy, and handle many of these and other tasks, thereby significantly reducing the frequency and amount of manual interaction required between man and machine. Figures 2.1 and 2.2 show how RBA can help automate the deployment of a Windows Service Pack, and provide two paths for follow-up: one upon the service pack's successful installation, the other upon an installation failure, both producing tracking data for a report on activities undertaken.

Figure 2.1: The RBA system can automate deployment of Windows Service Packs to selected target machines, then based on whether the installation fails or succeeds, report success or take remedial action, and then report on final results.

Figure 2.2: Ultimately, the RBA produces an audit report on the overall effects of the change requested—in this case, to install Vista Service Pack 1 on selected target machines—and provide information about the success or failure of the outcome, both in general terms, and on a per-target-machine basis.

Providing Auditable Records for IT Activities and Environment Changes

When change management disciplines are properly applied, resulting enterprise-wide coverage should be so complete that any single component change warrants instant notification to a central monitoring authority, and generates a comprehensive audit trail and change documentation. Thus, whenever and wherever support personnel implement changes to hardware or software subsystems within an IT infrastructure, some related audit alert or monitoring notice should be dispatched to an appropriate recipient, party, or process. This explains why so many RBA platforms incorporate some kind of audit-worthy record trails to track and documents IT activities and component or configuration changes, as shown in Figure 2.3.

Figure 2.3: As RBA tools push configuration changes, updates, and so forth, out to the network, they also record all such changes in a database, from which automated reports may be generated upon completion, or on demand.

In fact, using an RBA as part of change management can trigger updates to a configuration management database (CMDB) used to store configuration settings and to gauge real-time views of infrastructure changes. Such a repository of information catalogs all interrelated and unrelated components of an information system and represents an ITIL context to store, manage, and control authorized configurations for the significant components in any IT environment, as indicated in Figure 2.4.

Figure 2.4: When Network Administrators need to implement configuration changes, they can use an RBA system that integrates with a configuration management database to push changes out to the network.

As such, the CMDB is a fundamental component of the ITIL framework's Configuration Management process, and helps an organization comprehend relationships between components and track their configuration settings. The CMDB (and tools that use it, like RBA) also facilitates management analysis on the risk and impacts of unplanned changes, and how such changes can affect other resources on a network or system. In most implementations, in fact, the RBA system follows the ITIL-approved sequence for change management, as illustrated in Figure 2.5.

Figure 2.5: In the ITIL model changes are first designed, requested, and then built and tested outside the production environment. An RBA tool can integrate with all elements of the deployment phase, by automating change settings, pushing them out to target systems, verifying changes, and submitting those changes for final approval.

Empowering Frontline IT Early Responders to Handle More

Hands-on technical support is considered a must-have for any IT environment. That said, this does not mean that hands-on interactions or manual intervention is required to deliver such support, or to resolve problems or handle trouble tickets that emerge from handling support incident. Proper coordination with and use of RBA means that where most of a process or procedure can be handled in an automated, predetermined fashion, support staff need only trigger such responses rather than carrying them out manually. Inputs may remain constant in a variety of tasks related to IT infrastructure management, even if the outcomes are variable and unpredictable: RBA systems can manage this complexity, and help to ensure successful outcomes whenever possible, and to report problems or failures as and when they occur.

Whenever support personnel become engaged in manual routines, that effort takes away workforce power that could be available for other tasks and activities. Efficiency and effectiveness are products of how time and energy is spent in resolving an issues or responding to alerts. RBA enables support staff to spend less time in diagnosis and resolution, thereby enabling first-line IT and support operators to respond more quickly, and therefore to handle more alerts or incidents and reduce repeated efforts to battle recurring problems or issues.

Create a Consistent, Repeatable Change Management Process

Ultimately, the goal for any RBA system is to create a consistent, repeatable change management process that can be automated to create hands-free execution or implementation of manual processes and procedures within an enterprise IT landscape. Consistency and reliability are both instrumental to creating a practicable, self-sustaining management platform that encourages business to focus on achieving its long-term goals. Repeatability is essential for ensuring that recurring incidents, issues, and activities can be consistently and reliably handled each time they occur, whether by accident or design.

Establish Linkage Between ITIL Incident Management Processes

An intelligent and visually intuitive dashboard provides an excellent viewfinder into the network and system elements in the enterprise topology. Such a dashboard can combine complete ITIL incident, problem and change management visibility along with out-of-box reports that might include MTTR trending and cumulative ROI. An RBA system can chart the most frequentlyoccurring resolution workflows; it can also correlate alert and incident reports with ITIL configuration items and problems. Finally, RBAs support user-defined custom reports tor best suit each organization's unique reporting needs. The sample dashboard shown in Figure 2.6 illustrates data that includes service activities and various service quality metrics, including typical "speedo" style gauge readings at the center top of the image shown.

Figure 2.6: A well-designed dashboard brings metrics and information of greatest interest together for its viewers. In this illustration, the key point of focus is service quality; in other displays, it might be availability, end-user response time, update success, security scan results, or other items of interest.

For best coverage and interoperability, vendor-neutrality is an absolute necessity for sustaining a multidisciplinary enterprise network using RBA. Typical large-scale enterprise networks incorporate dozens of hardware architectures, operating across dozens of different software platforms. Individual platforms may be entire communities unto themselves—these may include virtualized or physical clusters, server farms, data centers, and so forth—but all must function together as a coherent whole. Although each individual platform requires individualized management and monitoring features, along with a secondary tier to manage and monitor activity at higher levels (such as for clusters and farms, or for various types of fail-over or load-balancing environment), it's also important to be able to weave all these threads together under a single point of view and control. From the RBA standpoint, these components all possess a variety of task-oriented capabilities that must be reworked into an overarching and comprehensive processoriented perspective at a higher layer of abstraction at the enterprise level.

An RBA platform should integrate such discrete, disconnected systems and disjoint management tools and processes beneath a single coherent and consistent umbrella. Adapters and other integration technologies that unite the RBA with other elements in the enterprise IT environment will usually be required to tie all of these disparate parts together so that just such a functional umbrella can be constructed and used for automation, monitoring, and management. In some terminologies, such elements are known as accelerator and integration packs, which may be defined as follows:

Accelerator packs: deliver usable collections of pre-configured workflows and operations for common IT infrastructures.
Integration packs: supply collections of pre-configured operations to interact with specific systems management tools such as provisioning, monitoring, or ticketing.

These kinds of RBA elements are designed to speed the implementation process, and to enhance productivity by providing pre-configured workflows and operations, Invariably these elements help enterprises address and automate workflows for common platforms such as Linux, Unix, and Windows, interact with widely used services (Microsoft Exchange, network load balancing, and database management systems), and work with key network infrastructure elements (routers, switches, firewalls, VPN servers, and so forth).

The Real RBA Payoff

Though obtaining a single locus for monitoring, control, and automation may be incidental to the process of deploying and using RBA technology, many enterprises find themselves realizing significant gains from its use. These often go well beyond the benefits inherent to automating repetitive or recurring tasks that result from eliminating human error, delivering documented change management processes, and freeing up human resources to work on other pressing and important problems. From the single and unified perspective that RBA delivers to enterprise IT, savvy managers and professionals soon realize they can approach problem resolution and future planning from a more business and service oriented perspective, and leave their fire hoses and axes in the firehouse where they belong. Although it's difficult to put a value on aligning IT goals and objectives with those of the business as a whole, it's not at all hard to appreciate the enhanced value and improved perception this brings to information technology and its operations, as end-users and executives alike begin to benefit from increased productivity, enhanced end-user experiences, and a less trouble-prone working environment.

In the next chapter, we will examine a scant handful of common RBA usage cases. In looking in more detail at event resolution, orchestrating and managing change, automating repetitive tasks and activities, and integrating multiple management tools and processes, readers will come to see and understand what endows RBA with its various value propositions, and how such systems actually look and behave in the workplace.