Is Gartner inflating the number of Leaders on Magic Quadrants?

SageCircle received an email from a reader asking whether we had seen the newsletter from a boutique analyst firm, which included a comment that Gartner has been increasing the number of Leaders on Magic Quadrants. The clear implication was that this analyst was accusing Gartner of corruption for inflating the number of Leaders in order to extract revenue from vendors in the form of analyst consulting days, research reprints, and so on. Of course, this analyst competes with Gartner for contracts and access to vendor briefings.

SageCircle has not noticed any “Leaders inflation,” but then we have not been doing any systematic, in-depth research which would be required for such an observation. We do note however that Gartner is being sued by ZL Technologies because the MQ that ZL has been listed on since 2005 still has only one (1) Leader and it’s not ZL. So I guess that Gartner gets criticized if it there are too few Leaders or too many.  The joys of being the dominate market player, everybody takes potshots at you.

The boutique analyst firm offered no proof, nor does it describe the research methodology behind the claim, so we cannot evaluate the validity of the claim.  Here are some general observations:

  1. The boutique analyst firm analysts could be looking at only a few MQs relevant to their coverage and these may have been around for a number of years. Maturing markets naturally see the vendors migrate up and to the right as the market consolidates through acquisitions or failures, vendors become better at execution, and so on
  2. The boutique analyst firm analysts do not notice that Leaders are not the only vendors who purchase reprints, vendors in all boxes – incredibly even vendors in the Niche box – acquire reprint rights and promote the MQs they are on. As a consequence, Gartner would not necessarily get incremental revenues because the Challengers and Visionaries might already be purchasing reprints of the Magic Quadrant
  3. We don’t believe there is any “Leaders inflation”

To see if the distribution of vendors around a MQ was skewed in one direction or another, we looked up a random set of MQs just to see what the breakdown was between the various boxes.  Our example set consisted of Continue reading

You don’t have to be a Gartner client to get a good “dot” on the Magic Quadrant

One of the continuing myths in the IT industry is that Gartner demands payment from vendors for placement on its research. This even came up in a comment – anonymously posted of course – on a blog post written by Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) called A Rant – My Integrity as an Analyst.

SageCircle knows this is not the case from personal experience, but also because we get collaborating evidence from our clients. Just last week we were on an inquiry with a client, a small software company, who was included on a Magic Quadrant in the Visionary square months before they even considered signing up for a Gartner contract. The reason for the inquiry with SageCircle? In the draft update of the Magic Quadrant their dot had moved to the left. Yikes. However, the reason for the less favorable position had nothing to do with their client status or the size of their contract. Rather it was because they had not noticed that the lead author on the Magic Quadrant had changed. Once we figured this out, they understood that their problem was that they had never briefed the new analyst.

We also know of large vendors who have spent hundreds of thousands of dollars with Gartner year in and year out only never to get onto a Magic Quadrant on which they wanted to be included.

However, in the past it has also been true that some unscrupulous Gartner sales representatives have played the research placement card when they desperately needed to Continue reading

So you were left off a Wave or Magic Quadrant – what next?

We don't exist according to ForresterIn January 2009, The Forrester Wave™: Community Platforms, Q1 2009 was published. This happened to be the inaugural publication of a Wave for this particular market. The primary author of this Wave was Jeremiah Owyang (bio, Twitter handle, blog) who conducted an incredibly transparent – for a Forrester or Gartner analyst – process for the creation of this Wave (see list of relevant blog posts at the end of this post). Jeremiah’s use of social media gave vendors in this nascent market plenty of opportunity to know what was going on with this research.

As is with the case with any piece of analyst research covering a new, dynamic, and extremely fragmented market only a fraction of the possible vendors can be fit into the time and space available. In this case, it was nine of more than 100 vendors. Neighborhood America, a social networking platform vendor, was left off the Community Platform Wave and did not take kindly to the exclusion. Neighborhood America created a web page, “Why weren’t we included in the Forrester Wave Report?”, to tell its side of the story. They also put a link on their home page (graphic above) to make sure visitors knew to go to the page. The explanation was reasonably well done (SageCircle would have counseled some additional text to provide context) and did not overtly attack either Forrester or the analyst. The latter part is important because an attack would have looked like sour grapes by a sore loser.

This was a very smart step to do. You can see the Community Platforms Wave graphic on Flickr and the PDF of the research note is easily available for free on the web. As a consequence, Neighborhood America prospects might see the graphic or Wave research and decide to drop them from a pending sales opportunity without further information. While Neighborhood America’s response will not get the breadth of readership that a Forrester research note will, it is a useful exercise.

SageCircle does not know the exact details about why Continue reading

Misunderstanding Magic Quadrants, MarketScopes, and More

Source: Gartner analyst blog by Lydia Leong

Source: Gartner analyst blog by Lydia Leong

For years IT and telecommunications vendors have complained about the misuse of Gartner Magic Quadrants by IT buyers.  It appears that three key issues are routinely surfaced: 

1)     The criteria for placing the dots onto the graphic are not transparent and often the dots appear to be randomly placed by the whim of the analyst

2)     Magic Quadrants are not always updated in a timely manner and out-of-date MQ’s seem to stay around forever

3)     Research consumers often look only at the graphic and miss the supporting research note or do not speak directly with the analysts via client inquiry.  This is especially true when free reprints are made available to non-clients by various vendors

Example Gartner disclaimer about the Magic Quadrant

Example Gartner disclaimer about the Magic Quadrant

Part of the problem is that while Gartner has background information about the MQ on its website (click here to read, free registration required) and a perfunctory paragraph to readers in the fine print in the footnote of MQ PDFs (click on graphic on left to enlarge), it does not have a systematically approach to training its clients about how the MQ is to be used. That is one of the reasons why SageCircle wrote IT managers, it’s never, ever only about the upper right dot when it comes to Forrester Waves or Gartner Magic Quadrants. (There is longer, more detailed version of this content in our  SageNote™ “A Consumer’s Guide to using Gartner’s Magic Quadrant”.)

It was therefore refreshing to see a blog post on the Gartner Blog network by Jim Holincheck entitled Misunderstanding Magic Quadrants, MarketScopes, and More where he talks a bit about criteria transparency and the way these reports should be used.  It makes a good read for both vendor clients and IT buyer clients.  This addresses the number one concern above. Perhaps with more discussion the use of these important tools can be improved.

However, there is still a disconnect with issue number three.  Jim states “More importantly though, Continue reading

“Prime the Feedback Loop” VP of Marketing’s excellent advice about Gartner

rocket-for-startups.jpgThis advice is just as useful for large vendors as startups

In Gartner for startups Michael Waclawiczek, VP of Marketing at expressor software, has joined the conversation started by Talend’s Yves de Montcheuil and Gartner’s Andy Bitterer (see Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant)  about startups ability to be included on Magic Quadrants.

Dr. Waclawiczek’s observations and advice are dead on and well worth reading. While directed at startups, his main points are applicable to large vendors as well. A quick summary:

  • For any vendor selling to high-end/large customers, dealing with Gartner is a given. Even if you decide to ignore them, your customers won’t.
  • At some point, you have to realize that the MQ is designed to meet the needs of Gartner customers – big companies looking for information, insights and backside-cover for big-ticket IT purchases.
  • My advice to fellow startups? Give up hope of making a real impact in “your” MQ, for now at least. But don’t give up entirely.
  • Work the Gartner system the best you can. Pull every lever you can reach.
  • Set your sights on Continue reading

Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant

gartner-andy-bitterertalend-yves-de-montcheuilThere is an interesting online conversation via blogs going on between Talend VP of Marketing Yves de Montcheuil (A comment on Gartner’s latest Magic Quadrant for Data Integration, photo left) and Gartner’s Andy Bitterer (Setting the Record Straight, photo right). This is interesting because it is unusual for a vendor to engage Gartner in a public forum about its research or methodology, and for a Gartner analyst to respond to criticism. Kudos to both Yves and Andy for engaging in this conversation. The other example that SageCircle knows of a vendor addressing perceived analyst shortcomings was by Jive Software Chief Marketing Officer Sam Lawrence. You can find a link to Sam’s post at Doing unto analysts what they do unto vendors. Sam received a positive response from the analysts to his post.

Yves basic complaint is that analysts like Gartner do not pay enough attention to open source vendors. He claims Gartner focuses too much on stodgy vendors that do not represent the future when researching the Magic Quadrant (MQ). Andy’s response is a point-by-point rebuttal of each of Yves’ comments including being very firm on their policy that vendors that do not meet the revenue criterion will not be included.

The Magic Quadrant methodology is not perfect when it comes to small vendors

While Gartner has steadily improved the MQ methodology over the last few years, it is far from perfect. One problem is that there is not complete transparency when it comes to all criteria, especially the ones based on the analyst’s subjective opinions. This is especially troublesome to small vendors without experienced AR professionals who do not realize they have to probe the analysts to get all criteria, their weights, and how they are scored. Another problem is that Continue reading

Executive sponsorship is critical success factor for a “Defending the Magic Quadrant” program

While we recommend that vendors don’t obsess over Gartner’s Magic Quadrant (MQ), it is also important that they realize that investing effort consistently between MQ refreshes will ensure that no nasty surprises pop up.

After we wrote in the SageCircle newsletter that vendors need to watch out for MQ complacency, especially if they are in the Leaders quadrant, Carter Cromley sent us the following email:

“We at SAVVIS are by no means complacent about our position.  In fact we’re incredibly paranoid to the point of having an organized “Defending the Magic Quadrant” program (that included executive sponsorship) that seemed to be effective in us maintaining our leadership positioning the web hosting MQ (just published).”

One of the key points that Carter makes is having Continue reading

Follow

Get every new post delivered to your Inbox.