End User Experience Monitoring as lynchpin for BSM

I get to have some very interesting and varied discussions with people interested in BSM. They generally start off with a high level strategic goal for their IT organization and go from there. A common theme that’s emerged in a large percentage of them lately is the topic of Application Performance Management (APM) and its relationship with BSM. More specifically, I’ve been spending a lot of time talking about End-User Experience Monitoring (as a subset of APM) as a key element of IT management strategy.

The most important factor when organizations look towards End-User Experience Monitoring (EUEM) technologies is of course that user experience is the ultimate success criteria by which IT services should be evaluated. If I could provide highly responsive application services that were always up and running, what else could be expected of me? The end users would be happy and the lines of business would be realizing the value of their application investments. The underlying technology at that point becomes largely irrelevant. Whether the applications run in corporate datacenters, are hosted by a 3rd party, run in a cloud, or are 100% virtualized, it just doesn’t matter. I am of course oversimplifying the matter by focusing on performance and availability of applications, while leaving out other elements such as security, but, for the sake of discussion, let’s stick to performance and availability for now.

The benefits for having an EUEM solution in place are obvious:

-- Instead of waiting for the phone to ring at the IT Ops Help Desk, you are proactively notified about application failures and brownouts (“this application is so slow!”)
-- Is the reported issue real; is the application really slow or is it performing the same as it always does?
-- Quickly get a handle on the scope of an issue; how many users impacted, which locations, is it all applications or just a few, etc.
-- Get initial diagnostic information to start the triage process; who gets called into the war room? Visibility always sounds great, but I always like to ask – “Have you considered the downside of total visibility into end user performance?”
-- Issues that previously went unreported will be obvious on the chosen EUEM products reports – If an application performs slowly and no one calls it in, was it a real problem? The answer is now Yes, whereas before the EUEM tool, it never happened.
-- Don’t be surprised when you hear talk about establishing internal service level objectives for performance, availability, and MTTR not for IT infrastructure elements, but End User Experience.
-- How well equipped are you to handle an increase in the number of events that need troubleshooting?  Now that you know every time an app fails or is slow, how quickly can you figure out WHY it happened?

This is where the BSM discussion starts to come up organically (unless it was the starting point, of course). Having a defined service model in place for the key applications & services that IT provides to the business is key to the WHY side of the EUEM equation. We talk about having a BSM implementation that ties together the best of breed End-User Experience Monitoring solutions, Service Desks, Fault and Event Management solutions, Element Management platforms, IT infrastructure performance monitoring tools, and other types of products to build a complete strategy for IT and Business Operations. Integration discussions quickly follow but that is a topic that has been covered extensively in other blog posts (but if comments suggest it’s time to revisit the topic, we certainly can).

I often wind up talking about specific types of EUEM technologies and vendors because of my background in this space (see full disclosure below) wanted to capture some of those discussions in a series of posts. First off, I generally group EUEM tools into one of three categories:

1. Passive monitoring systems

    -- Generally appliance offerings which capture and analyze IP packets to provide insight into End-User Experience of a wide array of applications
    -- May have specialized analyses for standards-based apps such as Web, Voice, and Video
    -- Deployment is usually at datacenters or wherever the applications are hosted
    -- Vendors providing solutions in this space (in alphabetical order):
            -- CA’s Wily Customer Experience Manager (via Wily)
            -- Compuware’s ClientVantage (via Adlex)
            -- Coradiant
            -- HP’s Real User Monitor software (via Mercury Interactive)
            -- IBM Tivoli
            -- NetQoS
            -- Nimsoft
            -- OPNET Technologies
            -- Quest’s Foglight End User Management

2. Active monitoring systems w/ synthetic transactions

    -- In most cases this involves recording sample user activities (e.g. login, search for info, run report, etc.) and using deployed robot agents/appliances to replay them at varied intervals.
    -- Most frequently these solutions tend to be oriented exclusively towards Web apps, but there are specialized vendors that focus on large enterprise application suites such as SAP and Oracle.
    -- Generally deployed at user population centers such as campuses or representative locations such as selected offices in Europe or Asia.
    -- Market was pretty much created by Mercury Interactive (now part of HP) so they are the dominant player here.
    -- Some of the vendors in the passive monitoring space also have some active monitoring elements in their portfolio to “poke” the applications when users might not be utilizing their applications (after business hours, for example); other smaller vendors offer lower cost solutions when compared to the HP suite including Managed Objects’ own Business Experience Manager.
    -- For internet facing applications, services from companies like Gomez and Keynote Systems are great sources of performance data without having to deploy your own monitoring “robots”

3. End-user behavior monitoring & analysis

    -- A few of the passive monitoring vendors offer aggregate or high level data, but I tend to group solutions into this category that have detailed analysis not just performance, but also business analytics about user behavior.
    -- Some of the defining elements in this category include capture and playback of complete user sessions, tracking time spent on each page, click-through rates, and error or missing content identification.
    -- Solutions in this space are specialized by applications (e.g. Web, SAP, Oracle, etc.)
    -- Deployment can be a combination of appliances and software agents. 
    -- A few of the vendors in this space (in alphabetical order):
            -- Aternity
            -- Knoa
            -- Tealeaf

The list of players in the EUEM space is not comprehensive, just a short list of those whom I have some level of familiarity. Some quick Google searches or a call to your favorite analyst should turn up more.

I would like to explore this area in more detail and plan a series of follow up posts with topics such as: details about what information from EUEM solutions should be incorporated into the BSM fabric (Hint: it’s not just events); sharing field experience on deployment strategies that have worked & pitfalls to avoid; detailed coverage on specific vendors including strengths, weaknesses, integration details, etc.

If you have experience with any of the products mentioned, other vendors that feel should be mentioned in any of the categories, general feedback, or would like to have a one-on-one discussion on this (or other) topic I’ve covered, leave me a comment below.

***Disclosure: The author of this post, Abbas Haider Ali, held roles at OPNET Technologies and IBM prior to Managed Objects.