CellarTracker Scoring Methodology

I am not a big believer in the importance of scores (vs. a narrative description of impressions of the wine), but if you are going to score, would your goal be to have a wildly different scoring range than other posters? And if so, why?

– Matt

I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others’ scores for a wine are completely not relevant to the score I give the same wine.

Of course not, but the OP’s point was not (I am assuming) to try to make each individual score match community scores for a given wine, but rather to try to make his range of scores match. That seems to me a good goal if you care whether your scores are meaningful to others.

– Matt

Well, that begs the question: Are “community scores” helpful/meaningful to others? There are many threads on this site, alone, wherein many folks emphatically answer that question with a “No.”

At what point does a collection of CT scores become a “community score” with which one should endeavor to match? Regardless of whether one’s TN is the first, tenth, one hundredth, or one thousandth TN for a wine, it still has just as much standing/merit as all other notes. If anything, the more recent notes should be valued more. Unless one would have the same worries about “matching” scores for a given wine with zero notes as they do for a gifen wine that has more than zero notes, it seems the whole notion of matching is logically flawed for the sole reason I put in bold, above.

A “community score” is all scores, averaged together. If, after some arbitrary number of scores, an user is merely looking to match their score to the community score then there is no point in scoring that wine, other than to further solidify/entrench the status quo.

It will never be possible to get everyone on the same page w/r/t scoring wines, or just about damn near anything, for that matter. Hell, look at Olympics, boxing, MMA, dog shows, cat shows, cooking competitions, writing competitions, Oscars, Grammys, Emmys, {anything where the scoring is, at least partially, subjective} and you will see that Opinions Will Differ. And differing opinions are to be celebrated and discussed, rather than denigrated and suppressed.

In my opinion, the best way to be helpful/meaningful to others with one’s TNs is to be consistent with oneself. The best way to be consistent is to have a strict procedure for assessment. I have a strict procedure for assessment; I feel I’m far more consistent/reliable since I’ve implemented this system than I was before using my system. Experience has also played a part in my consistency/reliability, but I do believe having a strict system of assessment plays a big role, too. As alluded-to earlier, however, I usually have not the interest or energy to go through such a strict process, which is why I don’t score most wines. … one kind of has to ask oneself when setting upon this Community Score Matching Path, “How much time and attention did others put into their scoring?” Is the tail wagging the dog?

Brian, it’s pretty easy to disagree. Someone who rates wines on a 1-3 point scale because they feel privileged to be different, should not have there scores averaged into a 0-100 point system. I am not saying those notes are not helpful on their own, but then they should, as you note, refrain from scoring or be open that their scoring is not based on any scale understandable by the other commoners using the site.

Yes that was the point, Matt. My goal isn’t to try and conform my own tasting notes to match the CT notes. It was asking whether my scoring system in the OP could be be refined, as I plan on refining it myself for greater consistency. I personally reject the idea that there is so little consistency in scores such that the average CT score is a random walk. My personal experience is wine that consistently gets notes of 94-95 is better in my own personal tasting than CT wines that score 85. One can debate where the line is, and it seems clear based on responses here that the refining I discussed in the OP is outside the bounds of error for CT (and I will take my score less seriously as a result). But that does not mean there should be no rules at all.

One thing I definitely took away from this discussion, is I am scoring average wines too high. I plan to rework my system per the recommendations, to lower my DPIM scores and be more willing to score wines a 88-89 (lowering my average wine to 88-90 instead of 90). I freely admit that I will score wines lower in the future than in the past, but (I) in the margin of error this is reasonable for the CT universe and (II) will allow me to be more consistent going forward. Barring a more refined set of guidelines from CT, this is a reasonable outcome.

My scoring is close to the CT scoring system but drops off quicker, though not as quickly as the OP’s “85 = Harsh/burning/acidic - don’t want to swallow”. From my experience, wines that WA or WS rate mid to high 80s are what I would call “Good”, not “Very Good” as in the CT scoring system. These are the wines that I might buy for parties or as cellar defenders depending on the price. Below 85 I would call below average.

Curious if there there is a wine rated low 80s on WA, WS and CT that would get a consensus as a “Good” wine.

We need to start a group who will adhere to the same guidelines when rating a wine. 85 is not a score that says do not swallow. That is a 70-75. But people don’t want to attach these low scores to a wine and then their user identity.

Joe, it would be nice but I don’t see how it’s possible. I do agree with what you’re saying though in terms of scoring. I’m thinking of a note I saw recently where the wine was ultimately described as “not good” and yet was still given 91 pts.

But that also hits on why I don’t score wines. I did in the beginning, as more of an academic excercise. But I I discovered quickly that few followed the methodology laid out by Parker or CT (I’d argue towards the end even Parker didn’t…). My scoring wasn’t comparable to others, and therefore couldn’t easily be translated. I also found that my scoring didn’t even mean anything to me…

In the end, it’s the notes that matter for me. My own notes as well as the notes of others whose palates seem to align with mine.

I agree with you, but I think we still are reading the point of this thread differently. To my understanding, everyone posting here agrees that individual scores on a given wine can and should vary from (distribute around) the community score, i.e., the average of the individual scores.

Personally, I am much more interested in the language of particular notes on a wine, at particular points in time, than I am in any individual score or in the average score.

That said, I and many others do read the individual scores, and if people are using them, I think that it helps for the scorers to be generally in line with a community norm as to what a given score means. Your notes, for instance, are very helpful (easily top 5% of the notes I read), because you give contextual information such as serving temperature and decant, and you are very specific and descriptive about aspects of the aroma, palate impression, etc. I don’t find your scores particularly helpful because (as far as I’ve followed) you compose them as an aggregate of a variety of subscores and I don’t know what those subscores represent, so in reading a number of your notes in passing I have never successfully calibrated what a particular number in a particular subscore might mean to me.

That’s all ok: I’m sure the notes are helpful to you and to people who understand your system, and you don’t have any obligation to make them helpful to me or anyone else in particular. But by using a non-standard system, you definitely make what you are saying more obscure, even if the intention is to be more specific.

Agreed, and agreed.

– Matt

I use a point system similar to some of the critics
Something like:
45 - minimum points just for being a wine
10 - color
15 - nose
20 - taste
5 - style for the varietal and location (example: a 100% steel tank 2013 Oakville cab would probably get a lower score)
5 - QPR

Don’t forget about ego, so many tasters on CT don’t want to admit they spent money on a wine they shouldn’t give 90 points to. [stirthepothal.gif]

I agree with this. However, I believe said conformity, or community norm, should be established up-front, rather than back-doored.

Your notes, for instance, are very helpful (easily top 5% of the notes I read), because you give contextual information such as serving temperature and decant, and you are very specific and descriptive about aspects of the aroma, palate impression, etc. I don’t find your scores particularly helpful because (as far as I’ve followed) you compose them as an aggregate of a variety of subscores and I don’t know what those subscores represent, so in reading a number of your notes in passing I have never successfully calibrated what a particular number in a particular subscore might mean to me.

That’s all ok: I’m sure the notes are helpful to you and to people who understand your system, and you don’t have any obligation to make them helpful to me or anyone else in particular. But by using a non-standard system, you definitely make what you are saying more obscure, even if the intention is to be more specific.

All very reasonable points, Matt. And thank you for the kind words. FYI, and for the benefit of others who also are confused by my system, I’ll happily share how I arrive at my scores. I use the Robert Parker scoring methodology. I believe CT promotes the same scale, if not the methodology in particular. As you’ve noted, my scores are broken-down into sub-scores — I do this for two main reasons: (1) discipline, and (2) information. As it was taught to me, the Parker scoring system works like this:

50 points just for showing up to the party as a liquid that is actually wine.
5 points for Color/Appearance/Body
15 points for Nose
20 Points for Taste/Structure/Palate
10 Points for Overall/Ability to Improve With Age

So, my TN scores are indicated as: Showing up, Color, Nose, Taste, Overall = Total Score
Breaking-down the score like this informs others of how I arrived at my final score; obviously, this won’t be informative if folks don’t know what categories I’m using, or what the total available points in each category are. I’ve found that analyzing a wine in such a manner requires discipline and focus, and leads to more thoughtful assessments. I’ve also found this system yields greater consistency, and that, in my opinion, is one of the most important attributes of a helpful taster. YMMV.

Yes, it would be great if we all analyzed, and scored, wines using the same methodology. That’s a big, and unreasonable, ask. But I do see the value in that hypothetical. I think we’re more in agreement than I previously thought. Thank you for your thoughtful posts on this topic, Matt. [cheers.gif]

I am not as trusting as others here about the equality of tasters experience or palates on CT. With thousands of tasters, given the usual Bell curve the scores should be weighted toward the less experienced I would think.
Another problem with taking CT scores at face value is a problem I have encountered more that once–2005 Freixenet Cava Brut Nature Vintage $ 8.64. Zero reviews but ranked as a score of 97 when searching
That might skew the " community average" little!

Is it my imagination, or do an awful lot of people simply point their wines on Cellartracker within 1 point of their favourite critic?

[winner.gif] And they use the same word salad to describe the wines.