October 31st marked the release of Wine Advocate’s annual issue reviewing Washington wine. Earlier in the week I discussed some highlights from this issue. Today I will begin one of several posts focusing on how Wine Advocate compares in a variety of different regards to its print peers Wine Spectator and Wine Enthusiast. My interest in doing so is educational, looking for similarities and differences. Suffice to say that while you and your children were sitting on the couch watching slasher flicks last Saturday night, I was slicing and dicing the Advocate, Spectator, and Enthusiast databases. For those who like numbers and data, sit back and enjoy. For those who don’t, my apologies in advance.
First up in terms of comparisons, I looked at the breadth of coverage of Washington wines. I took a ‘by the numbers’ approach looking at the overall number of published reviews and the number of wineries represented in these reviews. This approach obviously has some issues, mainly that additional wines are sampled but not published (more on this in a subsequent post). Other ways of looking at this, such as which wineries were reviewed/not, which AVAs were well-represented/not, would be interesting as well. However, I took this approach because 1) the data are readily accessible and 2) the others approaches mentioned require significant additional data, some of which is not available.
Looking at their published reviews, Wine Advocate and Wine Spectator have published reviews of comparable numbers of Washington wines in 2009. Wine Enthusiast, on the other hand, has published considerably more reviews. Wine Advocate’s published reviews also represent a smaller number of wineries than the other two publications. These data are summarized in Table 1.
|
Table 1 |
Wines Reviewed |
Wineries Represented |
|
Wine Advocate |
566 |
130+ |
|
Wine Spectator |
557 |
160+ |
|
Wine Enthusiast |
722 |
170+ |
Note that Spectator has two remaining issues this year and Enthusiast has four remaining issues, so their total numbers for 2009 are likely to increase whereas Advocate’s will not.
It is difficult to draw conclusions from this other than to say Wine Enthusiast has published more wine reviews and that the Spectator and Enthusiast have published reviews that represent more wineries. As I noted earlier, not all wines submitted for review are published, so the numbers above don’t speak to the whole universe of sampled wines and wineries, only the published ones.
Next, I compared how these three publications rated Washington wine in 2009. In doing so, I looked at the total number of wines reviewed and the number of wines that received 90+ point ratings and 95+ point ratings. I did this because the number of wines rated 90 points or above in the latest Wine Advocate seemed higher than what I had seen in Spectator and Enthusiast. Indeed, Wine Advocate rated a larger percentage of wines 90 points or higher compared to its peers. These data are summarized in Table 2.
|
Table 2 |
# of Wines Reviewed |
# of 90+ Ratings (%) |
# of 95+ Ratings (%) |
|
Wine Advocate |
566 |
341 (60%) |
30 (5%) |
|
Wine Spectator |
557 |
270 (48%) |
9 (1.6%) |
|
Wine Enthusiast |
722 |
281 (39%) |
11 (1.5) |
While at first blush it might appear that Wine Advocate is more likely to give a Washington wine a 90+ point rating than Wine Spectator and Wine Enthusiast, as I mentioned earlier, more wines were sampled than were published. So these data only speak to how each publication rated wines that they subsequently published. Doing a definitive comparison would require, for each, the number of wines sampled, the number of reviews published, and the ratings for the wines across the entire set. In other words, not going to happen. A different, interesting comparison would be to look at how each of these publications rated individual wines that they all scored (Sorry folks, not today. Maybe someday if I get very, very bored).
So to summarize, in 2009, Wine Enthusiast has published more wine reviews than its peers. Spectator and Enthusiast’s published reviews represented more wineries. Of the reviews published, Advocate has rated the highest percentage of wines 90 points or above with Spectator next and Enthusiast after that.
That is all for today. The next post on this topic will focus on differences in the publication rates and styles of each of these publications. As always, send along any thoughts or comments (Note: Please save the ‘get a life’ comments for personal correspondence).
First off-Great job on breaking down the numbers! My biggest complaint with the ratings is that the Spectator doesn't give Washington wines the proper respect. They throw in one or two in their "Other" wine states, lumping the best of Washington with Missouri, Delaware and the rest of those exciting wine countries.
And how much does the ratings in the publications relate to the amount of ad space purchased by the winery in the ratings? Just a paranoid thought, but I've heard it said before.
cityroute16, Thanks for the comment. I have actually put together some interesting information regarding the frequency with which Spectator publishes their reviews of Washington wine compared to their peers. Look for it in a subsequent post. Regarding ad space, I've heard this said as well, but have never seen any data to support it. That said, Advocate's lack of ad space is a nice differentiator for them in this regard.
Sean, very interesting breakdown – I've never seen anyone do it before. FYI – since I submit virtually all of the WA reviews to Wine Enthusiast, I can tell you that very few wines that are submitted to me do not get reviewed. The rare exceptions are 1) wines that have already been reviewed (I don't go back); 2) wines from really old, back-dated vintages; 3) wines with mysterious bacterial issues; 4) wines that are extremely limited, wine club only type releases; and 5) wines from brand new wineries that I have nothing good to say about (I like to give them a break). All told, maybe 80 or so wines a year would fall into one of these categories. A final note: the magazine publishes many more notes online than it puts in the print edition, so wineries should always refer to the online database when looking for reviews of their wines. Unfortunately, the magazine policy is not to post notes online until reviews are run in print, so there is often a considerable lag time between my submission of notes and the actual publication.
Paul, thanks for the comment and also for the background information (I was just about to hit you up for it actually). I have been corresponding with Jay Miller at Wine Advocate regarding the same type of information and am hoping to get information from Spectator as well. Once I have it all I will summarize it and include it in a subsequent post for reference.
Sean, thanks for the interesting post. Is it possible that the differece in 95pt+ percentage comes from what Paul said – TWA reviews all wines including very limited cuvees? To make it an apples to apples comparison, it would be interesting to look at scores for the same set of wines across publications, for example average scores given to specific lineups from high flying wineries.
Thanks for all the good work!
Ron, Thanks for the comment. Definitely not an apples to apples comparison here. Mainly just some interesting information regarding the reviews they publish. I will look to compare some of the wines the publications have in common at some point in the future. Very tedious work unfortunately. I would expect that it would mainly show some differences inherent in various reviewer's palates, but of course there is only one way to tell!