Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant

There is an interesting online conversation via blogs going on between Talend VP of Marketing Yves de Montcheuil (A comment on Gartner’s latest Magic Quadrant for Data Integration, photo left) and Gartner’s Andy Bitterer (Setting the Record Straight, photo right). This is interesting because it is unusual for a vendor to engage Gartner in a public forum about its research or methodology, and for a Gartner analyst to respond to criticism. Kudos to both Yves and Andy for engaging in this conversation. The other example that SageCircle knows of a vendor addressing perceived analyst shortcomings was by Jive Software Chief Marketing Officer Sam Lawrence. You can find a link to Sam’s post at Doing unto analysts what they do unto vendors. Sam received a positive response from the analysts to his post.

Yves basic complaint is that analysts like Gartner do not pay enough attention to open source vendors. He claims Gartner focuses too much on stodgy vendors that do not represent the future when researching the Magic Quadrant (MQ). Andy’s response is a point-by-point rebuttal of each of Yves’ comments including being very firm on their policy that vendors that do not meet the revenue criterion will not be included.

The Magic Quadrant methodology is not perfect when it comes to small vendors

While Gartner has steadily improved the MQ methodology over the last few years, it is far from perfect. One problem is that there is not complete transparency when it comes to all criteria, especially the ones based on the analyst’s subjective opinions. This is especially troublesome to small vendors without experienced AR professionals who do not realize they have to probe the analysts to get all criteria, their weights, and how they are scored. Another problem is that there is only so much room on a MQ and therefore only a subset of vendors can be included. This is also an issue with the Forrester Wave. Often the criteria for excluding vendors can be arbitrary and inflexible. Some analysts will include a sidebar or footnote mentioning interesting vendors who did not make the cut, but this is not a universal practice.

What could be the biggest issue for small vendors is the impact Gartner’s IT manager clients have on research. A significant source of information for the average Gartnerian comes informally via phone-based client inquiry. Often vendors hear Gartner analysts say “Well, I am not hearing anything about you from my clients” as part of the reason why they are not being covered. This reliance on end-user clients as a source obviously skews the pool of data points. Gartner’s client base is not a statistically valid sample for all types of research and only clients that choose to set up an inquiry are counted.

Small vendors need to take some responsibility

It is not just the flaws in the MQ methodology that exclude deserving vendors, they do themselves no favors by not seriously addressing AR. SageCircle has rather blunt questions for any vendor complaining about exclusion from analyst research like the MQ:

• Has the vendor truly invested in an AR program? By investment, we do not mean merely spending money on analyst contracts (though that is a tool), but have they hired an in-house AR professional and allocated serious executive and domain expert time to interact with the analysts?
  • A “jack of all trades” (e.g., PR manager or marketing director) might have analyst outreach as part of their task list but certainly does not have bandwidth nor skills (best practices) to engage in effective AR
  • Small vendors often rely on their PR agency for AR activity as well. This is often a major mistake as the typical PR agency is a pretender when it comes to AR expertise, often treating analysts like reporters a tactic which analysts loathe
• Has the vendor engaged in a concerted, long term campaign to influence the Magic Quadrant including at least one quality touch with each of the primary analysts every two to three weeks?
• Has the vendor talked to the analysts to understand the criteria for inclusion and placement?
• Has the vendor focused on influencing the underlying MQ criteria and not just try to get included on the MQ?
• Has the vendor tried to influence related research (e.g., Cool Vendors and Hype Cycle) that indirectly influence the MQ?
• Is the vendor effectively leveraging their customer base to educate the analysts?

Answering “no” to most of these questions means that the vendor will not be effective in influencing the analysts and the Magic Quadrant.

We do not have any information about Talend’s AR program, or if it even has one, but most very small vendors, open source or otherwise, do not have formal AR programs. As we pointed out in Are the analysts laggards or have startups neglected to brief them?, vendors should not be annoyed if they or their approach to a market is under covered by analysts if they have not invested the resources to effectively educate the analysts.

The size of the vendor really does not matter. What does matter is whether it sells to large enterprise IT departments, the core of Gartner’s and Forrester’s end-user client base.

SageCircle Technique:

• Small vendors that sell to large enterprises should ask their sales reps if they get push back from prospects because the analysts do not cover the vendor
• Small vendors should conduct a simple survey of their customers and prospects about their contracts with and use of industry analysts, especially for IT strategy and purchasing support
• Small vendors should seriously consider investing in dedicated AR headcount when there is evidence the analysts directly impact sales (confirmed by talking to sales and customers)

Bottom Line: Small vendors often complain bitterly about being ignored by analysts at major firms. However these small vendors rarely invest in appropriate AR. What is ironic is that small vendors often devote considerable resources to PR, both in-house and agency. If the vendor only shifted a small part of those resources to AR they would see hard ROI in terms of lead generation and increased revenues, something that PR cannot deliver.

Question: Analysts – What is the major barrier to your coverage of small vendors?

Related posts:

• Series on influencing Gartner’s Magic Quadrant. Part 1 is Don’t Obsess, Don’t Ignore: the Magic Quadrant & Tech Vendors and has links to all seven parts
• Category – Magic Quadrant (14 posts)
• Category – Startups (21 posts)

Advertisement