Research is always challenging, I’m required to stay objective, follow a consistent methodology, and inform the market who is strong and who’s not. Believe me, I’ve gotten my share of angry calls, emails, blogposts, comments, and conversations with brands and vendors as a result of my reports. I’ve found that one thing that really helps everyone to understand is to be open and transparent about the process.
In every Wave, like the Community Platform Wave I recently published, there are vendors that are elated and those that are disappointed, this is a normal outcome of reports that make the tough decisions to help brands make decisions. I’m empathetic to some of the vendors who were not in the Wave and want to make some clear explinations as they have to answer their colleagues, customers, and investors.
To start with, in a market of now 100 vendors, it’s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on my blog, twitter, indicating my intent as a result over 50 vendors submitted. We generally do not follow up unless we’ve heard client demand from inquiry calls, and I turned to 3 analyst peers for their inclusion of any additional vendors.
You can understand the challenges in filtering a large market to just a few vendors, given the amount of hours required to spend on each vendor, Forrester limits the number of candidates involved in the Wave, for this one, nine was a manageable size and we capped it at that number.
Next, we filtered vendors to meet the needs of external communities used for customers (I serve the Interactive Marketer) as well as vendors were primarily serving enterprise class companies.
Some vendors chose not to participate as being involved in a research report takes time of the vendors and perhaps precious time away from clients. Vendors have to participate in meetings, plan travel to meet me in lab, and be involved in several communications throughout the process. In fact, a multi billion dollar company who owns a community platform chose not to participate despite my offer.
I’m required to be objective, without this, I lose credibility and then everything else will fall apart. Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client. Despite this, I’m supported by my management, and our CEO to stay objective.
Now for the good news, while there is no immediate plans, Forrester is known for updating Wave’s in the space after a few years. Secondly, if you’re in this space, you know I publish a weekly digest giving updates to buyers about what’s happening in the market. I know for a fact the digest is read by senior decision makers at large corporations. If you are a vendor in this space, you can schedule a briefing with me and we can discuss your situation, I speak to clients, press, and media frequently and will always match market needs with vendors as deemed appropriate.
I believe in the open conversation, and will support the right for every vendor and customer to voice an opinion. I hope this helps provide insight to why some vendors did and did not make the Wave, we tried every effort to be fair in our first report in a very large market.
.
Jeremiah,
I applaud you for your integrity. I worked for Meta Group awhile back before they were acquired by Gartner and we took the same stance of objectivity in our research.
The calls you get from irate vendors in the market indicate a failure on their part to properly communicate their value.
I’ve always found it fascinating that companies will call up demanding to know why they were not included or why they were placed at a certain point in the report or study.
The questions they should be asking is of themselves – What value does our product have to the market and how can we effectively communicate that to everyone on a broad a stage as possible.
Again – I admire your integrity and your objectivity. To the vendors – quit your cryin’ and start communicating way more effectively.
Peter thank you, in a few cases, the lack of an AR strategy is to blame. I think there are some strong companies out there that did not get considered because they don’t get in front of analysts.
Hi Jeremy,
I don’t quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don’t hear your call, don’t follow you or don’t find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.
In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).
Lorenz
I didn’t expect you to read it that way, there was no bias at all. Please re-read my post, I suggest that it doesn’t matter if they are a client or not for the selection process. I’m sorry you misunderstood, was my wording not clear?
You’re right, following a consistent process is what I did. The vendor product catalog as a filter for a wave is a process that has been used for other wave reports, and in a market as large as this one, that’s certainly a key way for me to be efficient with my time.
To suggest that I reach to 100 vendors via email or phone would quickly overtake my normal duties.
Your second paragraph is spot on, you get the process.
Thanks for the thoughtful comment.
Sorry, I misunderstood the following paragraph: Several vendors who are in the community space who are Forrester clients were not included in the Wave, in fact, some were livid and gave me an loud earful and I actually suspect they may leave us as a client.
I guess the confusing came from the understanding of the word ‘Wave’. I now understand that ‘Wave’ is an end result.
The other paragraph:
‘To start with, in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog (free after registration). We used this information to filter out which vendors would be appropriate for the Wave report. I made a few public calls on my blog, twitter, indicating my intent as a result over 50 vendors submitted.’
lead me to assume that vendors needed to actively submit to the Vendor Product Catalog before they get included.
I am not an analyst and have only been following your blog for a short while. I will have to get used to some of the terminology and read the blog posts you reference in each post to fully understand the article in question.
Lorenz
Glad that’s a bit more clear. Yes you are right, submitting to the vendor product catalog was indeed a key requirement to be included to the wave. We needed that information to determine who would be filtered.
Then, I gave a caveat in the post indicating that I leaned on client requests to analysts (there are a few of us here) to see if we missed anyone. If so, we contacted them if they did not submit to the wave.
No surprise that you’re having to deal with some fall out. Thanks for the transparency. It’s much appreciated from our end.
“Now for the good news, while there is no immediate plans, Forrester is known for updating Wave™s in the space after a few years.”
At the pace that this industry is moving, that snap-shot in time that the Wave represents can quickly become outdated.
How long do you expect this study to be relevant and accurate? In some ways, I think parts are already outdated since we met in Foster City.
Michael
Whoah easy 😉
I appreciate the interest, but I need a vacation first before I can even think about it.
To date, a Wave update on this space is not scheduled for 2009, as I’ve already slated out my research agenda for the year.
It’s a lot of work –so an analyst can really only do it once they’ve had time to forget the amount of effort needed to complete it –the pain is still fresh in my mind.
Keep me updated of changes, the weekly digest will keep the market up to speed.
I totally understand. I only saw it from the outside and it seemed like A TON of work. I can’t imagine what went into it from your end. Respect.
My point is, is a large, one off report like this worth the effort vs what expanding on you’re doing here? Is there a different format (respecting a similar rigor) that can serve your clients better and is more scalable to have fast the market moves?
Don’t answer until you’ve had your vacation.
Re: updates, we announced App Studio on Tuesday. Integrating 3rd part apps to our application building software. http://tinyurl.com/9obyut
As one of the vendors (LiveWorld) in the top 9,of course we were happy to be included. I can also say that all 9 of the vendors took major issue with the wave elements they didn’t like. So this wasn’t some special club or easy ride even for the included vendors. Jeremiah held strong and didn’t change anything. We rated tops in services, but just strong in applications. This forced us to come to grips with his negative as well as his positive comments, whether we agreed with them or not. Enabling us to then advance in both services and applications. The moral of the Wave is, whether or included or not a vendor chooses to whether to learn or just to gripe.
Hi
Thanks for interesting reading (both author and others posting comments).
There is one thing I don´t quite understand though, and maybe one of you guys could help me to clarify this – what capabilities define a community platform? For me I have a bit of a hard time understanding the boundaries of such a product, and I would appreciate if somebody could enlighten me on this topic.
To clarify my quesstion – what makes for example LiveWorld or Jive a Community Platform, and lets say Microsoft Sharepoint or some similar product that has forums included – not?
James
No problem. It’s easy to identify, these vendors want to help create online communities with their customers for brands using social tools.
Hi
Not sure if I understand still, sorry, I still don´t understand the difference between products like Alfresco or MS SharePoint, that have social tools as part of the suite so to say, and the products described here as community platforms.
Let me clarify with an example:
A company is looking to expand their web portfolio with an online forum to get a community going. When thinking about a technical product to support this there are two ways to go – either leverage the current web site content management platform (assuming that the platform has forums, wikis and the other social media tools), or use one of the products that you define as community platforms.
So. The question is – what is the difference? Why is Alfresco, or Sharepoint etc, not a community platform, what are they missing in terms of functionality/capabilities compared to the products defined as community platforms?
Kind regards
/james
James
Ok, I think I understand better. This wave is for interactive marketers, so we found out which vendors were creating communities for customer-facing communities –not employee-facing communities. I expressed that in the 5th paragraph above.
Feel free to email me at jowyang@forrester.com if you have specific questions.
Jeremiah
Thanks for being patient with me 🙂
I did understand that we are talking about customer facing communities, as my example aimed at describing such a scenario, but obviously it was not clear enough.
In short – if you select a product for setting up a customer forum – why can´t you use Alfresco, SharePoint or whatever major CMS suite out there (the ones that have forums, blogs and wikis in their offering)? What capabilities do the products in your report have that makes them into a “community platform”?
What can, for example, Jive or LiveWorld do for me that, for example, SharePoint can not? What are the differences between a “true” community platform and a CMS that has forums, blogs and wikis included?
Let me know if this is getting tedious for you, was intented to be just a short question from the beginning. I am just trying to understand.
Kind regards
/james
Hi Jeremy,
I don't quite understand how you can give a balanced report if you exclude your own clients (I do understand that you are a stakeholder, but you are supposed to evaluate the BEST vendors, and you show a bias against your own). Earlier in the article you also mention that in a market of now 100 vendors, it™s very important that vendors take the onus to respond to the call for submission to the Vendor Product Catalog. Does that mean you are biased towards those people that don't hear your call, don't follow you or don't find your methodology the most ideal as it takes time away from their clients? It would appear that in essence it does.
In the end, in order to evaluate a product, you analyze the most prevalent needs of the market space, segment them into categories of needs, and test products in the marketplace in relation to those needs, then rank them per category. As long as you are transparent which clients are on your book, I have no problem with them being included in a study (in fact, they are right to be angry because they suffer an obvious disadvantage from the relationship).
sports shoes
Nike Sport Shoes
Women's Nike Sport Shoes
Men's Nike Sport Shoes
This is a great blog.