Lots of questions! Starting from the top...
Taste? Coffee cherries are mild in flavour and slightly sweet. The flavour is somewhat reminiscent of other red fruits, like a mild mix of fruits like raspberry, red mulberry, currant, cranberry, cherry, raisin. I've only had them dried, so they have a little bit of a "dried fruit" taste (e.g., as raisins are to grapes). To me, they also have a little bit of a mild taste of tobacco leaf. When steeped in water to make a drink called cascara (see this question/answers). The beverage tastes to me a bit like rooibos (red bush tisane). Some coffee shops sell the beverage.
Shipping? I've only seen dried coffee cherries for sale. Perhaps "fresh" coffee fruits are available in the coffee belt, but the seeds (i.e., coffee beans) are so valuable that I think the whole fruit doesn't make it very far outside of production regions.
Availability? Various specialty retailers sell the dried fruit pulp and skins (some call them the "husks") -- here's one example from Verve. Searching for coffee berries, cherries, husks, or for beverages like cascara or qishr may reveal more outlets.
Other preservation? Coffee skins/pulp dry well. Rehydrating is possible but underwhelming. I have tried to make a kind of marmalade out of them; see this seminal question, and also a follow-up question of mine at Seasoned Advice. I'm sorry to report that I have not had much success yet. The dried cherries are tough, so not pleasant to eat outright.
Good ideas! I really like the stuff, and am always happy to find others, and would be happy to hear more ideas about how to eat and drink coffee cherries.
EDIT: For completeness, here's a picture of dried coffee cherry pulp/skins that I use to make cascara.