For example, if I let a cup of coffee sit for 1-2 hours, would the amount of caffeine I get mostly be the same? Or does it decrease?
Caffeine is quite stable, it doesn’t evaporate or break down at room temperature (or coffee temperature), apart from dissolving in the water used to brew your coffee. So after an hour or two, the caffeine content is the same1 as right after brewing.
Unfortunately, the same is not true for some of the other substances that have us appreciate coffee, so you may expect the flavor to deteriorate a bit (or a lot, depending on whether you started with a handful of freshly ground slow-roasted beans or a few spoonfuls of Folger’s coffee-flavored sawdust). But that’s not what you were asking about.
1 If we were really nitpicking, we could even say that the caffeine ratio (not content) increases due to a minute bit of water evaporating during the given time frame. But for the average household precision, “stays the same” is sufficient.