Coffee philosophy: Ice then coffee or coffee then ice

Question

Ice + 2 Espresso Shots

OR

2 Espresso Shots + Ice

Settle the score please.

Answer 1

I'll give some background, a hypothesis, and (more important) instructions to find out experimentally.

The experiment is easy and fun. I did experiments such as paper vs. bamboo filter, slightly different grind sizes, fresh beans vs. 7-weeks off roast, and different coffee:water ratios within the range of a scoop measurement (i.e. is it important to weigh the coffee?).

Background: The Lady tasting tea experiment is at the start of modern statistics and randomized trials. (There's now a book, The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century.) The question was, “Does it matter to add tea to milk vs. milk to tea?” It turns out the answer is yes.

Speculation and Hypothesis: Adding milk to tea lets significantly more milk get hotter than adding tea to milk does. That could enable more of it to react chemically, e.g. curdling. But that's unlikely with ice, so I'll bet adding ice to hot coffee vs. hot coffee to ice makes no discernable difference in taste. (Let's ignore the fact that dropping ice into hot coffee could splash out more hot coffee than the reverse.)

How to Experiment: Do a Triangle Test with 3 or more people. Make the iced coffee both ways, "A" and "B", keeping all other variables as constant as you can. Each person gets 3 cups, one of which is different than the other two, e.g. a sequence like A-B-A or B-B-A. Give a different random combination to each taster, of course without revealing it.

All tasters should start by tasting their 3 cups in order. Then they can go back and forth between cups as much as they like. Their job is to try to discern the difference and say which of the 3 cups tastes the most different. You might also ask them to state their preference. When they all make their choices, then you can reveal the combinations.

The Math is Not Hard: If a taster gets it right, we really don't know if they were lucky or good. The default explanation ("null hypothesis") for the combined test results is that they're within a reasonable range of random results. Dice would randomly get 1/3 of the choices right. If enough people got it right that the odds were small enough of dice getting a result that extreme or better, then we'll conclude the tasters can discern the difference. Different fields of science may use different thresholds. Pick a threshold before doing the experiment. A common standard for "statistical significance" is ≤ 5%.

With 3 tasters, if they all get it right, that's a 1/(3³) = 1/27 = 3.7% chance, which beats the 5% threshold.

If K of N tasters get it right, the calculation considers the number of ways to get K of N correct choices and the probability of each random choice being correct (1/3).

The Google Sheets formula for Probability{x ≥ K of N correct choices} is:

=1 - BINOMDIST(K - 1, N, 1/3, TRUE)

so if that value is ≤ 5%, we'll conclude that people can reliably taste the difference.

Tip: To prepare for an experiment, I make paper strips with the different test sequences. Whoever pours for taster n will pick a strip at random, pour those cups while out of view, stash the strip in a pocket, and bring it out after everyone has written down their tasting choices. There needs to be at least enough strips for the number of tasters, and usually a multiple of 6 strips since there are 6 possible sequences. But when there are only 3 tasters, I prune it to 2 sequences that have one A and two B's and 2 sequences that have one B and two A's, so we can't all get the same odd one out.

Tip: Each taster gets 3 cups of coffee, a spit cup, water, crackers, and a napkin. I put painter's tape on the coffee cups to mark consistent fill lines and to label them 1, 2, and 3.