Review outlets need to stop pretending that tasting wines for review under different conditions doesn’t matter. It does.
There are dozens of outlets that review wine. Too few provide information about the conditions under which they do it.
Even at prominent outlets, it is common for reviewers to taste wines in a variety of settings. These might include mass tastings, where a hundred or more wines are tasted, at the winery, or at home.
Review outlets act as if these three settings are the same. They are not. These are radically different conditions that can produce substantially different results for the same wine.
To wit, ask any winemaker the setting in which they would prefer their wines be reviewed. They will almost universally say at the winery with the winemaker or another winery representative present.
However, those opportunities are typically reserved for highly regarded wineries. This gives those wineries a distinct advantage.
Wines reviewed at mass tastings, in contrast, are at a considerable disadvantage. Palate fatigue, tannin accumulation, and other issues can occur, even for experienced reviewers.
Tasting at home, meanwhile, a reviewer has considerably more time with a wine. This can, again, be a potential advantage for those wines.
Bottom line, tasting in varied settings can yield varied results, where some wines are advantaged and others are disadvantaged. Worse, for outlets that taste in non-standardized environments, the setting in which a reviewer tastes a wine is typically opaque.
What do the consequences look like? At Northwest Wine Report, I taste all wines under the exact same conditions. (Those conditions are detailed extensively.) However, I also taste at wineries. When I do, I take notes and score the wines, though these scores are never published. I have, however, compared them to my reviews under standardized conditions.
What I have found is that wines tasted on-site at the winery often, though not always, score better. In my experience, wines tasted at a winery typically score one to two points higher than the exact same wines tasted under standardized conditions. I have seen as much as four points higher.
How can this happen? It is certainly true that a bottle is only a snapshot in time, as is a reviewer’s perception. But I believe something else is going on.
When reviewing at a winery, the winemaker, or whoever is presenting the wines, is able to convey their excitement about the wine. Guess what? It works.
This can impact the reviewer’s perception and therefore his or her score. A winery could also decant their wine for hours or do any one of a number of things that might impact a reviewer’s perception.
Wine reviewing outlets need to stop pretending that tasting wines for review under different conditions doesn’t matter. It does.
How can this be fixed? It’s easy. Review outlets need to be more transparent about the conditions under which wines are tasted.
First, all review sites should describe in detail the manner in which wines are tasted for review. Second, sites that taste under varied conditions should specify the conditions under which each wine is tasted. This could be done by a simple icon or text listing.
That would allow consumers to decide for themselves whether the setting makes a difference or not. I can already tell you the answer: It does.
Northwest Wine Report needs your support. If you haven’t already, please subscribe. Northwest Wine Report is 100% subscriber funded. This site can’t continue to exist without your support. To those who have subscribed, thank you! You power this site.
Doesn’t this happen to us as consumers, too? I can’t tell you how many times I’ve loved a wine at a winery, only to open a bottle at home, let it decant, and wonder why it doesn’t taste nearly as good.
Mickey, this can ABSOLUTELY happen to consumers when they visit wineries, particularly if they are on vacation. It’s a great example.
Hi Sean, What do you do with the scores of the wines that you do not publish? In the times when you compare them to your past reviews under standardized conditions, do you use them to calibrate a different score that is then published? And, setting expectations with the wineries, they must be aware that they are hosting a tasting with you, yet the wines will not to be scored?
Gwen, I just file the scores that are tasted at the winery. I might use some of the information that the winemaker told me, such as vineyards or barrel usage, if I don’t have that information elsewhere, like a fact sheet.
The scores, however, are not used in any fashion, except that I do, oftentimes but not always, compare them after I’ve blind tasted the wines. At this point, I mainly do that to look at the “winery effect.” They in no way impact my scores from blind tasting. More than anything, I’ve found it interesting to look at. I should note that I never see examples of wines scored higher in my blind tastings than they were at the winery. They are always the same or lower.
I always make clear to wineries that if I taste with them, wines still need to be sent in for review in blind tastings. Despite the fact that I’ve been doing things in that manner for more than 13 years, many forget, given the large number of review outlets and varied manner in which people taste.
Interesting, thank you, Sean. You’d think the wineries would stop inviting you to taste if you need to taste the wines twice. That is a lot of everyone’s time, not to mention product! It makes me wonder about a winery’s sense of exceptionalism, as if there is always a hope that getting you (or any reviewer) on site, maybe just this once, will help the wines score better. As a writer, I can see that having a lot of dedicated time with the winemaking team can help get all the details you need to write about the wine, as the tech sheets usually leave out so much. Thanks again for the clarity.
The experiences of tasting at the winery can be invaluable. I certainly learn a lot, and tying that information in with later tastings is helpful. But, yes, it is labor intensive for all concerned. It comes with the territory though.
I also frequently do Zoom tastings, as my lack of ability to drive makes it harder to move around than it used to be. I’ve never checked to see if there is a “Zoom effect”! I’ll have to look at that.
I should add that there are wineries that require that their wines be tasted for review with the winemaker. Subsequently, I don’t review those wines, as it goes against my processes. My focus is making sure every wine get the same shot by tasting in a standardized way.