Is Gartner inflating the number of Leaders on Magic Quadrants?

SageCircle received an email from a reader asking whether we had seen the newsletter from a boutique analyst firm, which included a comment that Gartner has been increasing the number of Leaders on Magic Quadrants. The clear implication was that this analyst was accusing Gartner of corruption for inflating the number of Leaders in order to extract revenue from vendors in the form of analyst consulting days, research reprints, and so on. Of course, this analyst competes with Gartner for contracts and access to vendor briefings.

SageCircle has not noticed any “Leaders inflation,” but then we have not been doing any systematic, in-depth research which would be required for such an observation. We do note however that Gartner is being sued by ZL Technologies because the MQ that ZL has been listed on since 2005 still has only one (1) Leader and it’s not ZL. So I guess that Gartner gets criticized if it there are too few Leaders or too many.  The joys of being the dominate market player, everybody takes potshots at you.

The boutique analyst firm offered no proof, nor does it describe the research methodology behind the claim, so we cannot evaluate the validity of the claim.  Here are some general observations:

1. The boutique analyst firm analysts could be looking at only a few MQs relevant to their coverage and these may have been around for a number of years. Maturing markets naturally see the vendors migrate up and to the right as the market consolidates through acquisitions or failures, vendors become better at execution, and so on
2. The boutique analyst firm analysts do not notice that Leaders are not the only vendors who purchase reprints, vendors in all boxes – incredibly even vendors in the Niche box – acquire reprint rights and promote the MQs they are on. As a consequence, Gartner would not necessarily get incremental revenues because the Challengers and Visionaries might already be purchasing reprints of the Magic Quadrant
3. We don’t believe there is any “Leaders inflation”

To see if the distribution of vendors around a MQ was skewed in one direction or another, we looked up a random set of MQs just to see what the breakdown was between the various boxes.  Our example set consisted of

You don’t have to be a Gartner client to get a good “dot” on the Magic Quadrant

One of the continuing myths in the IT industry is that Gartner demands payment from vendors for placement on its research. This even came up in a comment – anonymously posted of course – on a blog post written by Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) called A Rant – My Integrity as an Analyst.

SageCircle knows this is not the case from personal experience, but also because we get collaborating evidence from our clients. Just last week we were on an inquiry with a client, a small software company, who was included on a Magic Quadrant in the Visionary square months before they even considered signing up for a Gartner contract. The reason for the inquiry with SageCircle? In the draft update of the Magic Quadrant their dot had moved to the left. Yikes. However, the reason for the less favorable position had nothing to do with their client status or the size of their contract. Rather it was because they had not noticed that the lead author on the Magic Quadrant had changed. Once we figured this out, they understood that their problem was that they had never briefed the new analyst.

We also know of large vendors who have spent hundreds of thousands of dollars with Gartner year in and year out only never to get onto a Magic Quadrant on which they wanted to be included.

However, in the past it has also been true that some unscrupulous Gartner sales representatives have played the research placement card when they desperately needed to

So you were left off a Wave or Magic Quadrant – what next?

In January 2009, The Forrester Wave™: Community Platforms, Q1 2009 was published. This happened to be the inaugural publication of a Wave for this particular market. The primary author of this Wave was Jeremiah Owyang (bio, Twitter handle, blog) who conducted an incredibly transparent – for a Forrester or Gartner analyst – process for the creation of this Wave (see list of relevant blog posts at the end of this post). Jeremiah’s use of social media gave vendors in this nascent market plenty of opportunity to know what was going on with this research.

As is with the case with any piece of analyst research covering a new, dynamic, and extremely fragmented market only a fraction of the possible vendors can be fit into the time and space available. In this case, it was nine of more than 100 vendors. Neighborhood America, a social networking platform vendor, was left off the Community Platform Wave and did not take kindly to the exclusion. Neighborhood America created a web page, “Why weren’t we included in the Forrester Wave Report?”, to tell its side of the story. They also put a link on their home page (graphic above) to make sure visitors knew to go to the page. The explanation was reasonably well done (SageCircle would have counseled some additional text to provide context) and did not overtly attack either Forrester or the analyst. The latter part is important because an attack would have looked like sour grapes by a sore loser.

This was a very smart step to do. You can see the Community Platforms Wave graphic on Flickr and the PDF of the research note is easily available for free on the web. As a consequence, Neighborhood America prospects might see the graphic or Wave research and decide to drop them from a pending sales opportunity without further information. While Neighborhood America’s response will not get the breadth of readership that a Forrester research note will, it is a useful exercise.

SageCircle does not know the exact details about why

Misunderstanding Magic Quadrants, MarketScopes, and More

Source: Gartner analyst blog by Lydia Leong

For years IT and telecommunications vendors have complained about the misuse of Gartner Magic Quadrants by IT buyers.  It appears that three key issues are routinely surfaced: 

1)     The criteria for placing the dots onto the graphic are not transparent and often the dots appear to be randomly placed by the whim of the analyst

2)     Magic Quadrants are not always updated in a timely manner and out-of-date MQ’s seem to stay around forever

3)     Research consumers often look only at the graphic and miss the supporting research note or do not speak directly with the analysts via client inquiry.  This is especially true when free reprints are made available to non-clients by various vendors

Example Gartner disclaimer about the Magic Quadrant

Part of the problem is that while Gartner has background information about the MQ on its website (click here to read, free registration required) and a perfunctory paragraph to readers in the fine print in the footnote of MQ PDFs (click on graphic on left to enlarge), it does not have a systematically approach to training its clients about how the MQ is to be used. That is one of the reasons why SageCircle wrote IT managers, it’s never, ever only about the upper right dot when it comes to Forrester Waves or Gartner Magic Quadrants. (There is longer, more detailed version of this content in our  SageNote™ “A Consumer’s Guide to using Gartner’s Magic Quadrant”.)

It was therefore refreshing to see a blog post on the Gartner Blog network by Jim Holincheck entitled Misunderstanding Magic Quadrants, MarketScopes, and More where he talks a bit about criteria transparency and the way these reports should be used.  It makes a good read for both vendor clients and IT buyer clients.  This addresses the number one concern above. Perhaps with more discussion the use of these important tools can be improved.

However, there is still a disconnect with issue number three.  Jim states “More importantly though,

“Prime the Feedback Loop” VP of Marketing’s excellent advice about Gartner

This advice is just as useful for large vendors as startups

In Gartner for startups Michael Waclawiczek, VP of Marketing at expressor software, has joined the conversation started by Talend’s Yves de Montcheuil and Gartner’s Andy Bitterer (see Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant)  about startups ability to be included on Magic Quadrants.

Dr. Waclawiczek’s observations and advice are dead on and well worth reading. While directed at startups, his main points are applicable to large vendors as well. A quick summary:

• For any vendor selling to high-end/large customers, dealing with Gartner is a given. Even if you decide to ignore them, your customers won’t.
• At some point, you have to realize that the MQ is designed to meet the needs of Gartner customers – big companies looking for information, insights and backside-cover for big-ticket IT purchases.
• My advice to fellow startups? Give up hope of making a real impact in “your” MQ, for now at least. But don’t give up entirely.
• Work the Gartner system the best you can. Pull every lever you can reach.
• Set your sights on

Vendor complains in a very public blog post about Gartner’s Data Integration Magic Quadrant

There is an interesting online conversation via blogs going on between Talend VP of Marketing Yves de Montcheuil (A comment on Gartner’s latest Magic Quadrant for Data Integration, photo left) and Gartner’s Andy Bitterer (Setting the Record Straight, photo right). This is interesting because it is unusual for a vendor to engage Gartner in a public forum about its research or methodology, and for a Gartner analyst to respond to criticism. Kudos to both Yves and Andy for engaging in this conversation. The other example that SageCircle knows of a vendor addressing perceived analyst shortcomings was by Jive Software Chief Marketing Officer Sam Lawrence. You can find a link to Sam’s post at Doing unto analysts what they do unto vendors. Sam received a positive response from the analysts to his post.

Yves basic complaint is that analysts like Gartner do not pay enough attention to open source vendors. He claims Gartner focuses too much on stodgy vendors that do not represent the future when researching the Magic Quadrant (MQ). Andy’s response is a point-by-point rebuttal of each of Yves’ comments including being very firm on their policy that vendors that do not meet the revenue criterion will not be included.

The Magic Quadrant methodology is not perfect when it comes to small vendors

While Gartner has steadily improved the MQ methodology over the last few years, it is far from perfect. One problem is that there is not complete transparency when it comes to all criteria, especially the ones based on the analyst’s subjective opinions. This is especially troublesome to small vendors without experienced AR professionals who do not realize they have to probe the analysts to get all criteria, their weights, and how they are scored. Another problem is that

Executive sponsorship is critical success factor for a “Defending the Magic Quadrant” program

While we recommend that vendors don’t obsess over Gartner’s Magic Quadrant (MQ), it is also important that they realize that investing effort consistently between MQ refreshes will ensure that no nasty surprises pop up.

After we wrote in the SageCircle newsletter that vendors need to watch out for MQ complacency, especially if they are in the Leaders quadrant, Carter Cromley sent us the following email:

“We at SAVVIS are by no means complacent about our position.  In fact we’re incredibly paranoid to the point of having an organized “Defending the Magic Quadrant” program (that included executive sponsorship) that seemed to be effective in us maintaining our leadership positioning the web hosting MQ (just published).”

One of the key points that Carter makes is having

Avoid like the plague – Using pseudo-Magic Quadrants in your analyst briefing presentations

After completing the in-depth Magic Quadrant series I was going to give this topic a rest for awhile. That is until I saw this tweet:

 

jowyang is the twitter handle of Forrester social media analyst extraordinaire Jeremiah Owyang. Jeremiah joined Forrester only last October and already he has seen so many vendors use a pseudo-Magic Quadrant that he is commenting on it. Can you imagine how bored and annoyed with this graphic other analysts that been around longer must be? I have seen pseudo-MQs that I swear were built on the idea of who is the leading vendor among left-handed IT managers who buy technology on Tuesdays in Guam.

Don’t get me wrong, I think the competitive landscape slide should be a component of almost every presentation made to IT industry analysts. There is no better opportunity to

Kleenex, Frisbee, and Magic Quadrant – what do they have in common?

Have you heard your spokespeople make the following statement when briefing the analysts or presenting to a group of analysts on a teleconference: “… also if I reflect on the way you put us, whether it’s your magic quadrants or …” Probably the executive was using “magic quadrant” as a generic label for analyst research graphics, much like people use Kleenex for facial tissue, Frisbee for a flying disc toy or Xerox for photocopying. 

Using Magic Quadrant as a generic label is dangerous for any vendor’s relationship with the analyst community. Analysts at firms other than Gartner bridle at Gartner’s dominate mindshare in the market. Referring to the Magic Quadrant is adding salt to their wounds. Gartner analysts, on the other hand, are extremely touchy about what they feel is the misuse of their signature research deliverable by the vendor community. So for vendors this is a lose-lose situation.

This situation also applies to other high visibility analyst deliverables like the Forrester Wave and Gartner Hype Cycle.

SageCircle Technique:

• Brief your executives* on the best practice for referring to

Equipping Sales for the MQ Effect: the Magic Quadrant & Tech Vendors [part 7]

Gartner’s Magic Quadrant can have a powerful impact on IT vendor sales cycles – anointing some vendors as a prime candidate for a sales opportunity while denying other vendors even a chance to bid. In order to exploit positive placement on a Magic Quadrant and mitigate negative placement, vendor sales executives need to work with AR to prepare and train their sales teams on certain basics about the Magic Quadrant.

To a large extent the Magic Quadrant is just another form of analyst research that can sales reps have to take into account when working with customers and prospects. However, the MQ does have some unique aspects that have to be addressed including: 

• Multiple MQs – A vendor can be on any number of MQs, which increases the chances that a prospect will be using wrong research
• Out-of-date MQs – Earlier versions of a MQ can be available for a long time, which can put a vendor with an improved position at a disadvantage
• Four boxes, four responses – How a sales responds to or uses a MQ is different depending

The Danger is Complacency: the Magic Quadrant & Tech Vendors [part 6]

There is a certain amount of self congratulations that occur when a vendor achieves a favorable “Leader” position on a Magic Quadrant. Because they are in the “Leaders” block, vendors feel like their job is complete. The problem is that such an attitude could lead to complacency and endanger a company’s coveted status in the future. Vendors in this situation could receive a nasty surprise as competitors leapfrog them or as they slip into the Challengers or Visionaries blocks.

This is not only a problem with “Leaders” since vendors in the “Challengers” and “Visionaries” blocks also feel that they can rest on their laurels. Most surprising are “Niche” vendors who are happy merely to be mentioned on a Magic Quadrant. The messages in this post are directed to Leaders, but also apply to all vendors, no matter what their position on the MQ.

What is the Danger? SageCircle had one client go from the best Leaders position in a Magic Quadrant only to slide to the Challengers block in the next version. Why? The vendor had become complacent about

Moving the Dot: the Magic Quadrant & Tech Vendors [part 5]

Repositioning your “dot’ on a Gartner Magic Quadrant does not happen just because you have a great product or service. It takes information, a plan, AR execution and avoiding mistakes. 

Expanding Your Goals – Moving the dot should not be the only goal of every analyst interaction. AR teams and spokespeople should insure that you accomplish your goal of moving the dot while working on other aspects of your analyst relationship such as competitive intelligence gathering, relationship building, training a novice analyst, strategy review, etc. Rarely will a vendor be interested in accomplishing one goal when interacting with the Gartner analyst in charge of a MQ. Some goals specifically concerning the MQ include:

• Moving your dot, either up or to the right or both
• Moving your competitors’ dots either down or to the left or both
• Increasing the distance between you and competitors
• Preventing your competitors from

Homework – Talk to the Analyst: the Magic Quadrant & Tech Vendors [part 4]

It is critical for AR to thoroughly research a particular Magic Quadrant and its history. Even AR staffs that have been working with Gartner on a MQ for a long time could benefit from doing a little digging into the background of the MQ in order to separate reality from faulty memory and myth.

SageCircle Technique:

Talk with the analyst - Obviously, AR should be interacting with their Tier 1 analysts on a regular basis on a number of issues. In many cases being in charge of a MQ means automatic Tier 1 status for a Gartner analyst. Many of those interactions will provide valuable insights into the MQ and the analyst’s criteria for it. However, there needs to be a dedicated call on the MQ* that occurs once a quarter. Topics to be covered include:

• Changes in the analyst’s responsibility, new additions to the team, both creation and peer review
• If you are on a Magic Quadrant with more than one author, what is the current

Homework – Gather Background Information: the Magic Quadrant & Tech Vendors [part 3]

It is critical for AR to thoroughly research a particular Magic Quadrant and its history. Even AR staffs that have been working with Gartner on a MQ for a long time could benefit from doing a little digging into the background of the MQ in order to separate reality from faulty memory and myth.

 SageCircle Technique: 

Check on past Magic Quadrants - The first task is to obtain past versions of the Magic Quadrants. You can search Gartner’s research database, but frankly you still have to ask. While Gartner analysts published dozens of distinct Magic Quadrants in the traditional Research Note format every year, there are so many publishing platforms at Gartner (e.g., presentations and toolkits) that a MQ can show up in either as an original piece of research or a reprint of something published earlier. Because not all

Common Mistakes: the Magic Quadrant & Tech Vendors [part 2]

For a variety of reasons, communications and IT vendor AR and executives make a number of mistakes concerning the Gartner Magic Quadrant (MQ) and how their companies should react to it. Decision makers at IT vendors need to take a step back and carefully consider the appropriate level of effort to put into “moving the dot.”

The first mistake is proceeding without understanding how your prospects and customers/clients value and use the MQ. You should be surveying your customer/clients and prospects about which research firms and reports they use.

The second mistake is assuming that you know what the underlying market-specific criteria and assumptions are for the MQ without talking to the appropriate analysts. Repositioning your “dot” on a Magic Quadrant doesn’t happen just because you have a great product or service. Often the most

Don’t Obsess, Don’t Ignore: the Magic Quadrant & Tech Vendors [part 1]

Even with the blogosphere and other forms of social media, Gartner’s Magic Quadrant remains the IT market’s most highly visible piece of commentary. Because the Magic Quadrant impacts billions of dollars of corporate IT purchases, some IT vendor executives put too much emphasis on “moving the dot” which drains resources from the overall AR plan. Other vendors decide to ignore Magic Quadrants, missing an opportunity to leverage an effective marketing channel. Neither approach is 100% appropriate. In this post, we provide background on the Magic Quadrant and suggest that vendors take a middle approach between obsession and indifference.*

It is not uncommon for a SageCircle strategist to hear the following comment from an analyst relations (AR) manager: “Our execs – or even board of directors – have made improving our position on the Magic Quadrant THE (not ‘a’) goal for AR.” While ignoring the Magic Quadrant (MQ) can be perilous to a vendor’s top line, too much emphasis on a MQ can drain scarce AR resources from influencing all the analysts covering your particular market. The downside is that AR won’t be able to develop counterbalancing relationships with analysts in other firms, leaving the vendor dangerously reliant on Gartner and the MQ for positive analyst coverage.

We think it’s time that vendors take

You could end up buried deep in the “Niche” by insisting on being added to a MQ before you are ready

Gartner’s Magic Quadrant is probably the iconic piece of analyst research. With its visibility and status, it also has enormous influence on vendor sales opportunities, especially when it comes time for IT buyers to draw up the all-important vendor short lists.

Because of this influence on short lists, communications and IT vendor executives sometimes obsess over the “MQ” for their markets and put great pressure on their AR teams to get placement on MQs they are not on. This can actually be a counterproductive move because vendors who really should not be on a MQ will get stuck in the lower left hand corner (click on graphic to enlarge). Even thought this part of the MQ is labeled “Niche,” too many IT buyers translate that label into “Loser.” Getting perceived as a “Loser” can put a vendor’s sales at a disadvantage in trying to get into an opportunity because it is so much more difficult to explain away why you are not a “Loser.” than explaining why your company was not included on a version of a MQ.

Some Gartner analysts publicly admit that this mental relabeling occurs and try to mitigate

Do your customers assume that Gartner or other analysts have done all the due diligence? [for Vendor Sales]

An analyst relations (AR) manager gave me a call this week with an interesting tid-bit that completely reinforces the recent postings about vendor sales reps asking about analyst usage, analyst myth #1  and how IT managers should use Waves and Magic Quadrants.
 
The AR manager was recently at their software company’s annual sales kick off meeting. There was a customer panel taking questions from a moderator and the sales reps in the audience. One question was “How or do you use the analyst firms to make decisions?” One customer said that

Now that is not the way to exploit an expensive Magic Quadrant reprint

I accidently came across an unsecured link to a PDF of Gartner’s Magic Quadrant for Team Collaboration and Social Software, 2007 courtesy of (vendor). This was a nice little treat because I was about ready to chat with one of (vendor)’s competitors and it good to see what Gartner had to say.

For (vendor), it is a classic mistake to not require someone who wants a copy to register so that the company can capture the potential prospect’s contact information for later use. Furthermore, Socialtext could be giving their competitors’ a free ride off their expensive reprint, because

IT managers, it’s never, ever only about the upper right dot when it comes to Forrester Waves or Gartner Magic Quadrants

One of the things that drives vendors – and even some Gartner and Forrester analysts – crazy is when an IT buyer zeros in on the vendors in the upper right hand corner of a Forrester Wave or Magic Quadrant to the exclusion of all other vendors. It is human nature to go for those who are perceived as tops in their market. Alas, that is not how these highly visible research graphics should be used. Rather IT managers should be looking to align their