to download a low resolution preview of this article - Roast Magazine

Laying It 

on the 

Table 

A Look at 

the State of Cupping 

by Andi C. Trindle Mersch 

As I said in the first half of this two-part “state of the coffee 

industry” series, “when you’ve been doing something for 

a while, you become compelled to step back and take a 

look to make sure you’re doing something worthwhile and that you’re 

doing it the right way.” This sentiment is especially true if you’re 

a perfectionist. Although I have grudgingly abandoned the idea of 

achieving self-perfection, I seem to still expect it of systems: iPhones, 

computers, cable TV—and cupping. Coffee cupping—as our industry’s 

formal and, thus far, only universally agreed-upon sensory qualityassessment 

tool that has very real financial implications—is a system 

that should be perfect. Right? 

By all accounts the industry has come a long way over the years, 

but perfection in cupping is probably as likely as perfection in me. As 

an industry, we may need to pursue the next evolution of cupping and, 

at the very least, we need some reflection and realignment. 

continued on page 26 

photo courtesy of Fonté Coffee 

24 roast September | October 2012 25

Laying It on the Table | Cupping (continued) 

Why We Cup 

Before we dig into some of the reasons for re-evaluation of the 

cupping process, let’s look at some of the reasons why we cup in 

the first place. According to the Specialty Coffee Association of 

America’s cupping curriculum, we cup coffees for the following 

reasons: 

• Purchasing 

• Quality and price discovery 

• Subject to Approval of Sample (SAS) applications 

• Quality assurance (incorporating roast profiling, as well as 

consistency monitoring) 

• Blending 

• Palate enrichment 

• Education/training 

These are some valuable motivations—particularly those 

that influence which coffees are bought and sold and at what 

prices. And, as Brian Aliffi, green coffee sourcing manager at 

Minnesota-based Caribou Coffee, states, “Cupping provides a 

simple tool for evaluating multiple samples in a convenient 

time frame and with a minimum required sample size.” 

Without spending any more time acknowledging the value 

and need for cupping, we can probably all agree that we need to 

have a formal, efficient system for evaluating coffees. But … 

Is the Cupping 

Process Working? 

photo by Chad Trewick 

Short answer: yes. More accurate answer: not wholly. 

As Martin Diedrich, founder of California-based Kean 

Coffee, says, “It’s so amazing to go to origin and be able to 

speak the same language.” This simple statement reflects 

what’s generally working with cupping. We have developed 

an industry language that allows us to understand each other 

across borders. This language of ours includes vocabulary and 

numbers—i.e., descriptors and scores—which theoretically 

help us agree on an objective coffee quality. From here, we can 

make informed buying and selling decisions that allow our 

businesses to succeed while, in turn, securing the growth of 

specialty within the greater coffee industry. 

Getting back to the longer and more realistic answer of “not 

wholly,” however, imperfections of the cupping process are 

many—from protocols to calibration to bias. 

Protocols 

“Protocols” is the term generally adopted within our industry 

to identify the best practice/industry guidelines for preparing 

a formal cupping. Protocols for consistency in roast degree, 

steeping time, grind degree, water quality and temperature, 

lighting, and many other factors exist to ensure that a cupping 

session is as scientific as possible, thereby yielding (in theory) 

objective results. Because coffee tasting is inherently rife with 

uncontrollable variables, such as the differences between tasters 

(taste buds, taste experience, psychology and physiology to name 

a few), it’s particularly important to identify and systematically 

manage these controllable variables. 

The SCAA, Cup of Excellence and CQI training programs, 

among other programs and companies in the coffee industry, 

have done an excellent job instilling cupping protocols around the 

globe in recent years. Although these organizations impart slight 

variation in protocols, they generally espouse similar guidelines, 

and all agree that consistent practices from session to session 

are critical. However, “in general, scientific practices are not 

consistently followed in most cupping labs,” says K.C. O’Keefe, 

founder of Café Verde Peru, a coffeehouse in Lima, and the chair 

of the SCAA’s professional cupper development committee. “This 

introduces unacceptable margin for errors in extraction and flavor 

development in the cup,” O’Keefe notes. 



Laying It on the Table | A Look at the State of Cupping (continued) 

Indeed, my own impetus for writing this article came from 

my frustrations owing to variant protocols in a lab setting. Earlier 

this year, I found myself participating in origin cuppings where 

some important, basic protocols weren’t followed. The intention to 

host a formal cupping was very clear, and these producers (unlike 

most around the globe) had appropriate equipment and the general 

setup for professional cupping. (Credit must go to dedicated NGO, 

industry partners, and government organizations around the world 

for devotedly dispensing cupping 

training and supplies in recent 

decades.) Nonetheless, training and/ 

or resources simply didn’t support 

meticulous analysis. In these cases, 

water temperature wasn’t monitored, 

roast degrees between samples were 

wide, and cuppers did not rinse their 

spoons from cup to cup. There were 

more problems than these, but these 

particular inconsistencies stuck out as highly problematic. Could 

I really make an informed buying decision or even provide helpful 

commentary about coffees where so many influential variables were 

uncontrolled? (Note: A buying decision on the ground at origin is 

separate from a final pre-shipment sample approval process.) 

I posit that many professional 

coffee buyers and sellers don’t 

maintain proper protocols 100 

percent of the time and some 

don’t even attempt them. 

To be clear, protocol lapses are in no way limited to origin-side 

operations. I posit that many professional coffee buyers and sellers 

don’t maintain proper protocols 100 percent of the time and some 

don’t even attempt them. 

In addition, some question whether existing protocols are 

sufficient and wholly accurate. As one example, Eton Tsuno, 

director of coffee at Sacramento’s Temple Coffee Roasters, suggests 

that water protocols should be published in exact gram weight of 

water in addition to traditional 

volume recommendations, since 

this echoes the current habits of 

baristas in measuring and evaluating 

espresso and filter drip extractions. 

The quality of water is even more 

important as water comprises 

approximately 98.75 percent of the 

brewed beverage in a traditional 

cupping. Despite the importance of 

water quality, however, labs around the world (and even within the 

United States) don’t use a standardized water base. 

For Christopher Schooley, a roaster who works with Coffee Shrub, 

a huge weakness of current cupping practice exists in the roast degree 




recommendation. “You have to look at more than one 

roast,” he says, because “there are problems with taking 

a qualitative measurement of both acidity and body, two 

things greatly influenced by roast development, unless you 

are looking at a couple of different roasts.” 

O’Keefe also feels that the sample-roasting 

specifications are too lax and instead requires that his 

samples are “roasted within 10:30–11:15 [minutes] rather 

than the eight- to 12-minute window of the SCAA.” 

Certainly, there are many other conditions and 

variables in cupping that might be better controlled or 

differently managed. Without postulating conclusions 

here, I suggest to the industry that we review existing 

protocols and best practices to see where we can improve 

upon them in the interest of achieving greater objectivity 

and balance of results. 

Of course, until everyone is rigorously employing 

protocols, creating more of them or adapting them doesn’t 

fix anything. 

Calibration 


Even if we had a perfect tasting process available to us 

and we employed its protocols religiously, a more difficult 

or does fruity mean cherry to some and over-fermented to others? 

challenge manifests in the concept of calibration. Calibration is 

Is an 85-point coffee in one lab at least in the range of 84–86 among 

the notion that, assuming equality of sample and process, cuppers cuppers of the same coffee elsewhere? For cupping to serve its critical 

around the world are consistently infusing vocabulary and scoring purpose as a buying and selling tool and price determiner, we have to 

with like meaning. Are the words citrus, fruity, savory and floral have semantic consensus among our professional tasters. As Tsuno 

applied consistently among cuppers of the same coffee, for example, states, “[Without calibration] pricing will break down, since in our 

small sector of coffee, price should 

be in direct relation to quality/ 

cupping score.” (Tsuno also suggests 

that “availability, sustainability and 

traceability” are part of the pricing 

equation.) If we aren’t speaking the 

same language, how do we determine 

fairly if a coffee is worth a differential 

of +50 versus +250? And, are we doing 

ourselves a disservice by selling coffees 

to consumers at prices they aren’t 

really worth? 

Over the years, I have frequently 

questioned whether we are universally 

calibrated as tasters. For example, 

many times, I have been witness to or 

part of debates over whether a “fruity” 

sample (in the same cupping session) 

is over-fermented and defective or 

whether it is a 90-plus-rated coffee. 

Well-respected and experienced tasters 

can land on opposite sides of this 

debate. 





Tsuno, having also experienced this 

both controlled and uncontrolled—to 

later in the article), but some of it is 

but who is right in these scenarios, and 

a generally poor reputation for quality. 

a deserved 88. Or, sometimes, because we 

particular argument, states, “I believe many 

find some objective end to these debates. 

because we don’t have universal language 

can there even be a “right” score? After 

Similarly, if a well-known industry taster 

feel that everyone else—a COE jury panel or 

people need more training in identifying 

However, even if we can come to agreement 

agreement. 

all, cultural consumer preferences are 

has scored a coffee highly, then others may 

a group of vocal roasters, for example—are 

uncontrolled ferment as a defect since what 

and can taste accurately whether fruit is 

Despite the frequency of these debates 

distinctly different and one community’s 

trend their own scoring on the high side. 

overrating a particular coffee, we are more 

we like about honeys and naturals is a result 

caused by happenstance versus deliberate 

over the years, to my pleasant surprise 

“good” is another community’s 

And, of course, the reverse is true as 

critical of it in an effort to assert our non- 

in controlled ferment.” Tsuno is exactly 

effort in processing, I suspect we will still 

when I questioned some fellow tasters 

“outstanding,” which warrant very 

well. A sample might be very good, but, 

bias. Allowing these influences to impact our 

right: as an industry, we need more training 

have variation on what is “positive” fruit 

about calibration, they were largely 

different scores. Still, with purchasing 

because a cupper is disinclined to believe it 

evaluation is, of course, unprofessional, and 

to calibrate. In this particular case, we need 

versus fruit that is “defective.” Some of these 

positive about their success in matching 

decisions and price points at stake, 

based on past experience, they score more 

we’d all like to believe we do not allow them. 

more training of processing manifestations— 

issues get into bias (which we will discuss 

their results with their outside partners. 

For Aliffi, Caribou finds that they “trend 

calibration has merit. 

The good news is that some level of 

conservatively, awarding it an 85 score versus 


in the same direction as our industry 

calibration among knowledgeable and 

contemporaries” more often than not. 

practiced tasters can happen. Tsuno 

O’Keefe asserted that his industry partners 

advises that “one should learn protocol 

“consistently score/calibrate within 

from an industry professional, and 

our results,” but he also acknowledges 

constantly review and communicate 

that they have some trading partners 

scores with other cuppers.” In particular to 

whose “results are very inconsistent.” 

scoring calibration, he suggests cupping 

For O’Keefe, these inconsistencies are 

“true commercial, 60- to 70-point coffees 

attributable to “counterparts neither 

and 90-plus-point coffees in order to put 

scientifically running the lab, nor 

80-point coffees into relation.” (Although 

consistently cupping,” though, and 

we have many discrepancies when looking 

not problems of semantic variation. 

at smaller score ranges, we are calibrated 

Aliffi also says that “for calibration the 

enough as an industry that this 10-point- 

greatest challenge is practices in the 

range categorization is generally agreed 

lab.” Without doubt, poor protocols yield 

upon and, therefore, a tasting can be 

result variation, but part of the problem is 

organized effectively.) 

semantic. We are not speaking the same 

For Caribou, Aliffi suggests that 

language—both in words and numbers. 

continued exposure to events through 

Scoring certainly shows itself to be 

the SCAA and around the industry—the 

widely varied among different tasters and 

likes of barista competitions, Coffees 

different groups of tasters. Obviously, 

of the Year judging, COE events and 

exact score matching isn’t possible or even 

Rainforest Alliance cuppings—“has 

ideal, but it seems important that we 

rounded our abilities and helped us 

are generally in agreement with a fairly 

develop and maintain our consistency.” 

narrow range of, say, two to three points 

O’Keefe echoes COE participation as an 

as a two- to three-point variation can 

important calibration tool, in addition to 

put a coffee into a different quality and 

“honest, open sharing of blind cupping 

corresponding price categorization. 

scoring/categorization with your trading 

This group score just manifested itself 

partners.” 

in our office as we hosted a tasting of the 

Nicaragua Cup of Excellence (COE) coffees. 

Generally speaking, the entire group of 

Bias 

more than 12 outside tasters, plus our 

four cuppers, ranked the various lots with 

scores ranging from 82 to 90 and generally 

a solid few points (or more) lower than the 

COE jury panel across the board (all jury 

scores were above 85). Protocol variation 

like roast degree might very well have 

been different and the influence of time 

between the jurors’ evaluation and ours 

was likely impactful (as an agricultural 

product, green coffee’s properties will 

change over time regardless of sample 

preparation and evaluation methodology), 

One of the most prevalent problems with 

the cupping process is the influence of 

bias. Fortunately, many biases are readily 

apparent and relatively easily mitigated. 

For example, if a cupper knows that a 

particular sample is from a producer who 

pays scrupulous attention to quality or 

from a farm with an excellent reputation, 

they may be inclined, without conscious 

awareness, to score it higher than they 

would score a coffee coming from an 

unknown farm or from an origin with 



However, even among experienced tasters, 

these biases can occur despite best intentions 

and firm belief that we are in control of 

our partialities. Human psychology is very 

powerful. 

So, cup blind. This simply means, cup 

without knowing what you’re cupping. Have 

someone else set up your tasting sessions 

ideally using codes only and don’t even try to 

figure out or guess what you may be tasting. 

Just taste and evaluate honestly and with 

focus. Blind cupping isn’t always possible or 

practical, and sometimes there are distinct 

advantages to cupping with knowledge of 

what you are tasting—calibration training, 

for example. Nonetheless, as participants in a 

process with results that impact pricing paid 

to farmers and feedback that can influence 

their future care of their beans, we should 

control the controllable variables. 

The Next Phase 

Ultimately, I feel that the cupping process 

is a worthwhile and important one, but 

it’s a process fraught with imperfections; 

the current cupping process will never 

achieve true objectivity. Of course, we 

don’t necessarily need complete objectivity; 

there are cultural preferences and biases 

that create a need for subjective variety in 

the marketplace. The key as professionals, 

however, is in having a system with as much 

impartiality as possible. As specialty coffee 

tasters, it can be easy to think of our job 

as just choosing the very best coffees from 

around the world to showcase, but we should 

remember that our decisions at the cupping 

table have real financial implications for 

hard-working coffee producers around the 

globe. So, knowing the limitations of the 

cupping process, as Schooley recommends, “it 

would be considerably helpful for [buyers] to 

look at coffees a couple of different times and 

in different iterations before making a final 

judgment.” Maybe we can all agree to do at 

least that while we re-evaluate and improve 

cupping evaluation in the long run. 

Andi C. Trindle Mersch is a green 

coffee trader with Atlantic Specialty Coffee, Inc. in 

California, where she also runs the quality-control lab. 

She has been working in specialty coffee since 1989. Andi 

has volunteered with the SCAA Training Committee since 

1995 and joined the board of directors in March 2010. She 

can be reached at atrindle@ecomtrading.com.

to download a low resolution preview of this article - Roast Magazine

Create successful ePaper yourself

Delete template?

Save as template?