How to use analyst market share numbers after Gartner makes a “huge mistake” with server market share numbers

Rarely do analysts call out another firm on perceived failures in research, but Rob Enderle does just that in Liars, Damn Liars and Statistics: Gartner Goofs on Server Numbers. Money quote:

“…However, the accuracy of these numbers even inside corporations (given how deals are accounted for) would suggest that getting within 5 percent of actual sales would be very difficult, let alone having a high level of confidence that under 1 percent actually signified real market leadership. …”

Rob then goes into an interesting discussion of the shortcomings of market share numbers and the methodologies used to create them. The article is well worth reading. It would be interesting – fun even – if more analysts engaged each other in the marketplace of ideas rather than having a monologue with clients.

SageCircle has long said that market share numbers from the market research analysts can provide interesting insights into the direction a market is going. However, relying on the numbers alone without

People on the Move in Analyst Relations

Claire Dessaux, VP of Analyst Relations, Oracle

Skip MacSkill, Director, Global Analyst Relations, Nortel Networks

Melissa Selcher, Director, Analyst Relations, Cisco

Please join me in congratulating these folks and wishing them great success in their new positions.

Are you or someone you know on the move? Please let us know and we’ll post the news in future editions of People on the Move. Send us the information to info [at] sagecircle dot com.

Why analyst relations matter – Analysts do not have time to do all-inclusive research

(After an interesting Twitter-based conversation with Illuminata’s Gordon Haff and former IDC analyst Ida-Rose Sylvester over the use of the word comprehensive, we have decided to use the word all-inclusive instead. )

One aspect of the analyst industry that is not widely known by technology buyers (aka end users, usually IT managers) and vendors is that industry analysts do not have the resources (e.g., time and travel budget) to conduct and publish comprehensive all-inclusive research about a market.  Advisory analysts gather most of their data from client inquiry and vendor briefings.  The major firms do not conduct product evaluations, lab tests against specifications, or quality of service investigations.

 This point was highlighted by Forrester analyst Jeremiah Owyang in Starting the Forrester Wave: White Label Social Networks and Community Platforms about some research he is working on:

 ”…I made a call for the vendor product catalog in this market, (and via email and twitter) that document is a detailed index of over 40 vendors in the space, (aprox 50% of the market) and will be available to Forrester clients…”

 ”…Due to the rigorous methodology … The Wave will only include several vendors.”

There are two key points here, one is that the vendor catalog is only a subset of the market and, two, the Wave will be a further subset of the vendor catalog the analyst assembled.

For vendors in this market these points should send a shiver down their spines. If they

Another top social media analyst leaves Forrester, this time Peter Kim

Well, well, Forrester loses another social media analyst inside of a week, this time Peter Kim (blog, Twitter handle). Unknown at this time whether Peter and Charlene leaving (see here) is a coincidence or the beginning of a trend.

Remember, most analyst firms have not invested in knowledge management systems so most information that analysts get in vendor briefings and other sources is stored between the ears of an analyst which means that it walks out

Right services – Align the services you buy to better match the reason for info or advice [Purchasing Analyst Services, Part 3]

(Based on comment’s Forrester VP Eric Lobel and review of notes and Forrester quarterly earning call transcripts, we are changing this post to remove Forrester from the discussion that the move to role based research is a means to significantly raise the price of syndicated research. While Forrester executives do regularly talk about raising the average selling price of its services through reduction of discounts and annual price increases, there is no price difference between WholeView and RoleView.)

Forrester and Gartner has have a variety of services that they offer at different price points. One of the products that both firms are Gartner is pushing their its sales forces to sell more of is the role-based products (“RoleView” at Forrester and “Gartner for Business/IT Leaders” at Gartner). During its 1Q08 earnings conference call Forrester’s CEO even introduced a new metric, “roles per client,” for financial analysts to track. Gartner’s CEO updates financial analysts each quarter on the progress his firm has made in switching clients from traditional Core Research seats to the role-based seats.

Why the emphasis? Switching a client from Forrester WholeView or Gartner Core Research to one of the Gartner role-based seats is effectively a significant (up to 100%) price increase. The draw is for the additional “analysis” more suited to the person’s role.  While a role-based seat might offer sufficient incremental value to be worth the price difference for some buyers, that might not always be the case. 

It is important for buyers of analyst services, whether enterprises or vendors, to carefully examine all the deliverables associated with

Analyst firms should notify vendors about staff changes

Especially vendors with scheduled briefings, consulting days, or key projects by analysts  who submit their resignations

Analyst relations (AR) professionals are sometimes blindsided in the final preparations for a long scheduled briefing, analyst summit, or analyst consulting day (aka SAS) to discover that the analyst had submitted his or her resignation several weeks before. Worse yet are situations where the vendor has just conducted a briefing only to learn days later that the analyst has just left the firm. Either way it is bad for AR who now has to scramble to change plans and could experience the wrath of executives who perceive that AR just wasted their time by being uninformed.

For a variety of reasons, analyst firms are reluctant to admit that an analyst is leaving the firm. However, these reasons are insufficient for withholding critical information from AR teams who work hard to facilitate the flow of information from the vendor to the analyst firms. It is not appropriate for the firm to arrange a last minute substitution without

Why analysts matter – “I get asked daily in one medium or another who to buy”

Some analyst relations (AR) managers are lucky in that their executives really get the analysts and their impact on the vendor’s leads and sales deals. Alas, not all AR professionals are so lucky. However, there is a resource to use to educate* executives about the impact of the analysts – the analysts’ own words. For example, here is a throwaway line by Forrester analyst Jeremiah Owyang in Starting the Forrester Wave: White Label Social Networks and Community Platforms:

          “I get asked daily in one medium or another who to buy”

Jeremiah is very good about keeping vendors and end-user clients alike up-to-date on what he is working on via his blog posts. This particular line was not bragging, but explaining one purpose of the Forrester Wave, which is to help technology buyers develop their short list of vendors to invite to a bid. Because it was not the main purpose of the post, I think that makes it even more powerful education tool as it

Top social media analysts leave Forrester and IDC

News is that IDC Research Director Rachel Happe (Twitter handle, blog) and Forrester VP & Principal Analyst Charlene Li (Twitter handle, blogless for now) are leaving their firms.

These departures are big blows for both firms, though in different ways. Rachel was really IDC’s sole expert on social media so her departure eliminates a big piece of IDC’s intellectual property on this market. Forrester has a team of social media experts, so expertise is not the problem. For Forrester, Charlene’s departure is a loss of prestige and credibility as Charlene is one of the most highly visible experts in this market. Obviously neither loss is a fatal blow to either firm,  but they need to work to fill the holes quickly.  

(Added 7/3/08 at 5 am PT. Links to Rachel’s and Charlene’s blog posts)

Leaving IDC…Joining Mzinga Rachel

Why I’m Leaving Forrester Charlene

Here is an update on Charlene via Twitter “Looking at options, likely will be on my own with a combo of blogging, speaking, consulting, and influence building”

Remember, most analyst firms have not invested in knowledge management systems so most information that analysts get in vendor briefings and other sources is stored

Killer questions to ask PR agencies to see if they are AR pretenders or contenders

There are many drawbacks to using PR agencies to conduct analyst relations – not the least that most analysts hate dealing with agency staff.  Sometimes, however, communications and IT vendors have no choice but to farm out some analyst-related activities. To minimize the chance of agency staff causing problems with the analysts, vendors need to carefully evaluate whether or not a PR agency is actually competent in analyst relations before hiring them.

A technique SageCircle has developed is asking a series of questions in the form of scenarios about AR situations. The responses to the scenarios can then be graded for compliance with AR best practices and insights as to how the analysts work. As always, it is important to weight the questions because some will be more important than others. In addition, it is critical that a standard evaluation framework be established so that responses from different agencies will be graded consistently.

The killer questions should not just be asked of the agency’s senior executive that is trying to win the business, but also the staff that will actually be doing the work.  Reluctance by the agency to introduce you to the staff should raise red flags about the breadth and depth of AR expertise in the firm.

The first killer question to ask the PR agency rainmaker and staff is

Examples of analysts using blogs for research purposes

As we pointed out, analysts are increasingly using blogs as research development platforms so monitoring analyst blogs is a good way for analyst relations (AR) to get insights into analysts’ work-in-progress. With this information in hand, AR teams can then decide whether to join the conversation online or reach out to the analysts for a briefing or inquiry.

 Because relatively few AR teams are monitoring analyst blogs, those AR professionals that use this technique can achieve a competitive advantage by getting in early on developing ideas when they can have the most impact.

 Here are two recent examples of analysts using

Why AR is more important than PR for a Startup [Startup Saturday]

Research by SageCircle, H&K (in multiple Technology Influencer Studies conducted by Penn, Schoen & Berland Associates), Lighthouse AR and other AR advisory groups has consistently shown that the most significant influence on purchases is peer recommendation and personal contacts.  Second is industry analyst opinion, which leads all other influence including advertising and PR.  For a startup attempting to break into an existing market or carve out a new market space this is critical information.

For a startup, traditional PR is certainly important and should not be ignored, but allocation of resources to AR can provide a higher ROI.  Press is very transient and even an outstanding article or mention does not have staying power over the long run if it gets buried in the clutter of a Google search.  Analyst reports have a much longer shelf life and may be referred to months after they are published as a relevant research note is more likely to surface during a research search on the analyst firm website.  Good research consumers will then contact the analyst firm for an update or discuss the report during an inquiry.

Industry analysts also convey information at industry events, act as sources for reporters, and can even have influence on Wall Street.  It is therefore critical that they are