As part of my blog redesign, I have added more information about how I taste and review wines at Wine Enthusiast. Please note that this does not represent any change in practices. Rather, it is a further explanation of the practices I have been using since I became a contributing editor at the magazine in 2013. You can read information about the rating system used at Wine Enthusiast as well as information about and special designations here.
All wines are sampled blind and in a standardized setting
All wines that I review for Wine Enthusiast are sampled blind in a standardized setting (location, time, serving temperature, and stemware). I am in all cases unaware of the producer, appellation, vintage, and price when sampling and reviewing wines. Doing so is an attempt to remove biases that such awareness can cause, both positive and negative.
While I regularly taste wines with winemakers at home and at wineries, these notes are strictly informational. All scores come from bottles sampled blind, in a standardized setting.
There is no screening of samples prior to evaluation
All wines that are part of my beat (currently Washington and Idaho) that are submitted for review are subsequently tasted and reviewed by me personally. No one else screens or reviews my wines under any circumstances.
Unlike some publications or reviewers, there is no screening process for wines prior to being submitted or reviewed at Wine Enthusiast. I am looking at everything that comes through, rather than a subset.
The benefit of this is that it allows me to taste broadly across wines that are part of my beat, providing both myself and consumers with a clearer picture of what is going on in the area. The benefit for wineries is that it eliminates gatekeepers when submitting wines.
Scores for all wines submitted for review are subsequently published*
The benefit for consumers of not screening wines prior to review is that it allows them to see almost all of the scores given to the wines that I taste. If a wine is submitted to me for review, a score is subsequently published for that wine.
When wines are screened, the scores by definition represent a subset of the wines submitted. This means, for the consumer, when you don’t see a review for a particular wine, you never know, was it not submitted? Was it submitted and screened out by someone before getting to the reviewer? Could the winery just not get their foot in the door to get someone to even screen the wine?
If a wine was submitted to me for review at Wine Enthusiast, I will taste the wine, and you will subsequently see a score published. There are only two exceptions. The first is if the wine was rated less than 80 points. We consider these wines to not be commercially acceptable. The second is if only one bottle of a wine was submitted and it was determined to be contaminated by TCA (cork taint). These two reasons are why there is an asterisk (*) above.
The process
Sets arranged by variety/style
The capsules are stripped from the wines, and the wines are placed in opaque bags prior to evaluation to obscure identifying information. Wines for review are sampled in small sets of four to twelve. Sets are arranged by variety or wine type, such as a Bordeaux-style blend or Cabernet Sauvignon. I do not sort wines by and an unaware of when tasting producer, vintage, appellation, or price, again to remove the possibility of bias that this awareness might bring.
Small sets and no mass tastings
I keep the number of wines I taste on any given day small, typically from a minimum of eight – the most common number of wines I sample per day – to a maximum of 20, with the latter being rare and typically only done for white wines. I avoid tasting larger numbers per day as I believe doing so is not fair to the wines or their producers.
Tasting large numbers of wines per day can lead to a variety of issues, for example tannin accumulation, that can unduly influence a score. Additionally, when tasting large numbers of wines in a single setting, even the most skilled taster who is faithfully spitting all of the wine they are sampling will be influenced by the effects of alcohol.
I have used a breathalyzer to confirm this during previous (non-Wine Enthusiast) mass tastings I participated in, where I have registered above the legal limit after a morning of large-scale tasting (60+ wines in three hours), even though I was spitting all of the wine. This effect is compounded by a morning and an afternoon of mass tasting (often well over 100 wines). These mass tastings are not uncommon at wine competitions and also professional review tastings at some publications.
During mass tastings, a reviewer’s perception for the first wine they taste is obviously vastly different from the 40th or 100th due to the effects of alcohol. I believe this is unfair to all of the wines in question. Tasting in small sets removes the potential effect of substantial alcohol absorption on the subsequent scores that can occur in large scale tastings.
Multiple passes per wine
When tasting wines for review, I typically make more than one pass through the flight, taking independent notes each time. I usually spend 5-10 minutes per wine, tasting the wine and writing a tasting note. These tastings take place over several hours.
While I may taste a wine over several days to get a sense of its overall ageworthiness – and I resample all wines on the second day to look for latent cork taint – any notes beyond the first day when the wine was tasted blind and in a standardized setting do not affect the subsequent score. Rather, they might affect the wine’s drinking window if it is a Cellar Selection and perhaps the other special designations listed here.
Evaluating typicity and overall quality
When tasting wines for review, I am evaluating them for typicity and overall quality. Typicity means, for example, does a Washington Merlot taste like a Washington Merlot? If it does – or does not – this may subsequently impact the score or text of the review. For example, if a Washington Merlot smells like a very green Cabernet Franc, this would be considered atypical for the variety and would most likely impact the score.
Quality looks at a number of factors, such as the pleasantness of the aromas and flavors as well as the overall balance, complexity, depth, intensity, finish, and length. I also think about what a wine is trying to accomplish and how well it accomplishes it.
I try as best I can to review wines irrespective of my own personal palate preferences. That is to say, I am trying to assess what the quality of the wine is, not whether I might personally like to drink it at home. This differs considerably from how consumers approach wine.
For example, let’s say a Chardonnay is made in a ripe style and fermented in 100% new oak that has an overt, but balanced, influence on the wine. Perhaps I do or do not personally care for that style, but, regardless, if the style is well executed, I would rate the wine highly. For a consumer, if the wine is not in a style that they would personally prefer, they would most likely pass on it and not think highly of it. This is fundamentally different from a critic’s approach.
If you have any questions about how I taste wines for review at Wine Enthusiast, please email me at [email protected].
Image by Richard Duval.
Sean, very nice article. So insightful to understand how wines are evaluated. Thank you for taking the time to prepare this article. – Chris
Sean – it's a nice article and nice to see that some reviewers are tasting blind. But I have to quibble with the idea of "typicity". What exactly is a Sagrantino from Washington supposed to be like? Or Ugni Blanc?
"Typicity" means you're comparing against some standard, usually someplace in Europe. But if you believe in the concept of terroir, the wine should taste like it comes from wherever it's from, not like it is a poor imitation of a wine from another continent.
Monovarietal wines should not be the standard. Nor should the first person making a wine in an area get to set the standard by which all subsequent wines in the region are judged.
And then what if it's a blend of Trousseau and Pinot Noir? Or Nebbiolo and Grenache? How can either of those be typical of anywhere? They may be wonderful, but without a benchmark, you'd have to rate them merely on their own merits, which seems like a good way to rate any wine.
Finally, I know that to people, terms like Washington and Idaho are meaningful. But to plants, those are irrelevant. I would have no problem if you were to taste wines from a more logical area, say the Columbia Valley, which would make more sense and seems to be how the regions of France are assigned to reviewers. The idea that in the US you taste by state, and when it comes to Spain you taste by wherever they speak that language (!) makes no sense. Obviously I know it's not your decision, just mentioning it. And no knock to Michael either.
Anyhow, nice article in general. Best of luck!
Anon 1/14/20 10:44am, indeed, comparisons are only useful if there is something to benchmark to. And again, typicity is only a part of the evaluation and not a particularly large part honestly. But it can become important if a wine is way outside the bounds of what is typical for the variety in Washington and throughout the world.
For example, let’s say a Merlot is overtly green, the example I gave in this article. Or a Cabernet has a lack of tannic structure or a Petit Verdot seems to be missing acid or tannin, things that are hallmarks of each variety the world over. Now I might still give the wine a good score if it possesses other factors that still make it a delicious and high quality wine (the old yum factor), but I would certainly mention it in the tasting note at the very least. When knowledgeable people are buying a bottle of Cabernet from Washington, they expect something. If a bottle is markedly different from that and they are buying it based on my score or recommendation, I want them to know that.
In fact, from the 2016 vintage, I have had several Merlots that I rated highly but noted that they drank a bit more like a Cabernet Sauvignon or Bordeaux-style blend because of the herbal influence, which was not uncommon in this vintage (cool finish, larger crop). In this case, the score was not impacted, but the tasting note was. I’ve had other wines in the past where I have written in the tasting note, “It doesn’t taste like FILL IN THE VARIETY, but who cares? It’s delicious,” and scored the wine highly.
But then I’ve had other Merlots from 2016 that veered very far into the green, such that they bore little if any resemblance to the variety that was on the label. It was very far outside what I would expect or a consumer would expect from that variety. There both the score and the tasting note were affected. Would I feel differently if it were a variety that tends to have a bit more greenness, like Cabernet Sauvignon or even more so Cabernet Franc or Carménère? Perhaps then the score would have been impacted a bit less, as those qualities can be more expected in those varieties.
The idea of typicity gets more difficult the further away you get from varieties that are common in Washington or elsewhere in the world. In fact, it can get so far away that the concept becomes meaningless. There are obviously a number of varieties being produced in Washington that lack reference points both inside and outside the state. This is part of the enjoyment and challenge of reviewing wines in this area.
When that’s the case, I revert to evaluating the wines on a more basic level: looking at the aromas and flavors as well as the other factors that I mentioned that influence quality. At its most basic without any point of reference it can be, “How enjoyable is this wine as a red/white wine?”
Similarly, for blends, which you mentioned, I simply have the categories, “Red Blends” and “White Blends.” I don’t get fussy about what is in them. I just worry about how high quality the wine appears in that broad bucket. As I stated in the article, I also think about what is the wine trying to accomplish stylistically and how well it accomplishes it.
As with anything, the more data points you have, the more meaningful that data can be. For Cabernet Sauvignon, for example, I’ve tasted thousands of Washington Cabernets. I might still taste one unlike I’ve ever tasted before (and do on occasion), but I would certainly at the very least mention this distinctiveness in the tasting note.
CONTINUED
CONTINUED
Conversely, I am looking forward to tasting my first varietal Clairette Blanche from Washington later this year. This lacks a reference point in the state, and I will simply be evaluating it as a white (or in this case sparkling) wine. Will all subsequent Washington Clairettes be evaluated against that as a standard? No. But it’s beginning to build a mental database of the wines that might at some point become meaningful. Or not! Reference points only become interesting once one is established, not necessarily by an individual wine but by a group of wines. But they are not always established, and in that case typicity bears little impact.
In terms of the concept of terroir, I agree. Wines should taste like where they come from. But typically on any wine there is both a varietal influence and a growing region influence, and honestly ideally both are apparent. For example, a Merlot tastes like Merlot but also tastes specifically like a Washington Merlot. In rare areas, like for example the Rocks District, the area can even trump variety or at least be in front of the variety. And that’s fine. I wouldn’t, for example, ding a Grenache for tasting more like the Rocks District in some respects than it does Washington or world Grenache. In fact those distinctive qualities can be part of what make it so interesting. Typicity is just a piece of information in the equation.
In terms of area, I agree wholeheartedly. This is part of the reason that I review both sides of the Columbia Valley, Walla Walla Valley, and Lewis-Clark Valley. Growing region is obviously more important than observing a state line. I would love to see the same be the case for the Columbia Gorge, but that wasn’t my decision.
Thanks for the comment! Hope this gives some additional perspective. I might use some of my comment as a follow-up blog post follow up on the subject.