Over the years, there have been wild claims about various (or most) coffee packagers adding MSG to either their coffee or the packaging, supposedly to make their product more addicting or at least more popular. People react negatively to such claims for many obvious reasons -- they've heard that MSG is harmful, they don't want adulterated coffee, they don't want to be duped or manipulated, etc. There has never been any actual evidence that any coffee packager actually did this, and lab tests disproved the claims when specific coffee companies were targeted.
Some retailers now put "No MSG" notices on their coffee offerings for the same BS marketing reasons they put "gluten free" on products that never contained gluten. So to the extent people are aware of the disinformation campaigns or see the "No MSG" notices, they are likely to frown on it as someone adulterating their food.
I suspect the negative reaction to hearing that you do it is either the same generalized reaction (if it's bad for coffee packagers to do it, why would you do it yourself?), or the fact that it's an unusual substance to add to coffee. For example, some people are aware that adding a small amount of salt to coffee can enhance the taste (if you like that); the people who aren't aware of it often consider it strange when they hear about it. And adding MSG is far more unusual. Some people are adventurous, and anxious to try new things. Others cringe at the idea of things that sound unusual.
You may have jumped to a conclusion that there must, or could, be a good reason not to add MSG to coffee based on the often negative reaction people have to the idea. The negative attitude can be explained, but I'm not aware of factual or compelling supporting reasons not to do it (other than the person believing that small amounts of MSG are harmful).
