Can the IT industry analysts be objective?

question-mark-graphic.jpgI recently saw this question on Twitter. It was from an AR practitioner, but it also applies to research consumers as well. Several studies* of IT manager clients surfaced that “objectivity” or some variation is an important consideration for buying analyst services. But is this a reasonable expectation for the research consumers? The answer is a qualified “yes” and there are ways for analyst clients to ensure that objectivity is as high as possible.While it is our opinion that analysts do not pull their punches because vendors are clients (see Analyst integrity issues – the urban legend that won’t die), that does not mean that analysts are 100% objective all the time. Every person has life experiences that will permanently or temporarily make them a little less objective.  AR teams may have their doubts about some analysts - but they are indeed human.  Their bias can show when answering a client’s inquiry or writing a research note. Reasons include:

  • Background – Analysts can come to their current jobs after being an IT manager (like me), a vendor (usually product management), a reporter, a firm client service staffer, a college student or from some other career. The analyst’s background can significantly influence his or her thinking, especially soon after joining the firm. This bias fades over time.
  • Philosophy – People can have a philosophical bias for or against a particular industry trend (e.g., open source software). This bias can lead to the so-called religious wars where both sides of an issue will vehemently argue their position without listening to the other side.
  • Demo glow – There are times when a vendor technology demonstration will be so effective that analysts will temporarily become infatuated with the product or technology. This state of mind usually fades within days or a few weeks.
  • Effective outreach by vendor – A vendor can bombard an analyst with lots of content, executive access and attention and effectively shape the analyst’s worldview and opinion of other vendors.
  • Ineffective outreach by vendors – In the same vein, badly conducted interactions or neglect by AR teams can leave key analysts with negative feelings or lack of information

* Studies: InformationWeek, SageCircle, Analyst Strategy Group, Hill and Knowlton

SageCircle Technique: When conducting an inquiry, research consumers – whether IT managers or vendors – should tactfully probe the analyst’s thought processes to ensure that subconscious biases are not coloring the analyst’s recommendations.

  • During an inquiry with an analyst they do not know, research consumers should ask the analyst about their background and how long they have been with the firm.
  • During an inquiry, ask the analyst what would cause them to recommend a course of action 180 degrees different. If the analyst says “nothing,” then philosophical bias might be present
  • During an inquiry, ask the analyst if all the vendors being covered in the conversation have been equally effective in keeping the analyst briefed. If the answer is “no,” probe to see where the analyst is getting information about vendors who are doing a less effective job of briefing the analyst
  • If the analyst seems really positive about a particular product – and it is the type that can be demoed – ask the analyst whether the vendor had recently demonstrated the product for the analyst. If the analyst gets rhapsodic in their enthusiasm for a recent demo it is possible the analyst is in the grip of demo glow

Bottom Line: IT analyst clients have to use good research consumer techniques to ensure maximum value from their analyst services contracts. One technique to use is to determine whether any subtle biases are affecting the analysts’ research and recommendations.

Question: Research consumers – Do you consistently check to ensure that the analysts are as objective as humanly possible?

Are you getting the most from your analyst contracts? SageCircle can help. Our strategists can:

  • Evaluate the usage of your contracted analyst services and suggest ways to maximize business value from your investment
  • Train your colleagues with analysts seats (e.g., Gartner Advisory and Forrester Roleview) through efficient and effective distance learning via webinar or teleconference
  • Critique your upcoming analyst contracts to ensure you are getting the right services from the right firms to meet your business needs
  • Save you time, money and aggravation

To learn more contact us at info [at] sagecircle dot com or 650-274-8309.

About these ads

3 Responses

  1. OK we’ve done “independence”, and now “objectivity” – probably next there’ll be “balance”, “comprehensiveness” and the like. I’ll try to keep this short, just in case I need to repeat it later:

    ANALYSTS ARE HUMAN BEINGS

    No really, they are. This absolutely isn’t a dig, Carter, more something I just wanted to state for the record. And also, for the record, I think we’d rather it were true – that analysts were human beings, warts and all – than the alternative, that there really did exist some technological caste of perfectly formed high-priests that had somehow got a monopoly on the truth. Well, perhaps there are some analysts that believe… but anyway ;-)

    Don’t get me wrong. Analysts are in a highly privileged position, at the mid-point between vendor and consumer of technology. In such a position, when it works, it is possible to do great things. It is equally possible to abuse the position, and we need to have appropriate checks and balances in place to ensure advice is as well thought-through, objective and independent as possible. The most important of these, in my mind, is transparency: as Alan Pelz-Sharpe has said, different analyst models are not necessarily wrong – but (me talking now) abusing the position most certainly is.

    But we’ll never be able to hide our humanity, and nor should we. On my blog, I mention a penchant for voice recognition and shiny gadgets. As a person, I loathe spending money unnecessarily, but will occasionally convince myself that a certain purchase is necessary, despite all evidence to the contrary. Do such characteristics make me a bad analyst? Potentially – but here’s the rub – it is such human traits that make us all the rounded characters we are, rather than mindless, shortlist-generating automata. We have a appropriate due process to ensure that by own, undoubtedly biased world views are not the ones that come to be presented on the printed page. I can remember back at Quocirca a few years ago, when I was advocating blogging, when everyone else was highly dubious. I was right about the potential; they were right about the timing.

    Trouble is, in this business I do think we have a tendency to stick certain people on pedestals – the term “expert”, when I was in a proper job, always used to be defined as the person that didn’t manage to duck fast enough when a difficult issue came sailing through. Once on a pedestal, it can be tempting to start to believe your own mantras – at which point the learning process comes to a shuddering halt. I’ve been guilty of it myself in the past, but not for very long as invariably something would happen to burst the bubble shortly afterwards!

    I do understand the importance, however nebulous, of “gravitas” – which is at the root of the ability of an analyst (or anyone else) to advise on a subject and be taken seriously. However, I believe that analysts should equally be able to demonstrate their humanity, which is a core element of the analyst interaction. A great example of this is Redmonk, a firm which is as much about the characters as the (sizeable) capabilities. Its an interesting case study – as I understand it, Microsoft is a Redmonk client in part quite because the Redmonk guys won’t just agree with everything that Microsoft says – that’s the whole point! I’d like to think we are the same, sometimes it is only by telling some hard truths that we can add the most value.

    Meanwhile – absolutely – we should strive to be independent, objective, balanced and comprehensive in our coverage and advice. But lets not throw out the quite human baby with the bathwater. To repeat – transparency is the key.

  2. […] up of bias and getting paid to say how awesome some client is in these things. I always feel like that kind of question is like being asked the old “so how long has been since you’ve stopped beating your […]

  3. Thanks for the RedMonk plug there, Jon. It’s all true – of course! ;)

    On the topic, I think it’s always handy to hunt down anyone’s motivations. I do it all the time – “sizing people up,” as some put it. I like to think I make it easy for people to size me up when they meet me online or in-person (though I can be quite the mopey introvert sometimes), which I hope lets people sniff out my bias or no.

Comments are closed.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: