They aren't just frothing the milk, they're also stretching the milk. Stretching it is what gives a latte its unique texture (though the micro bubbles add to that). If you want a good latte, focus first on stretching it - you can use a small whisk to get air into the milk if need be.
You want a small stainless steel pitcher, most look like this:

Put 2/3 of the amount of cold milk you actually want in your latte in that, then put it on a heat source (small burner is okay). Hold your hand around the pitcher while it heats, and use the other hand to stir the milk. You can also use one hand and just swirl the milk around in the pitcher - do what feels comfortable, just keep the milk moving.
As soon as the pitcher becomes just less than too hot for your hand, pull it. Swirl it around a little bit and look inside. Does the milk have a kind of gloss to it similar to Elmer's glue (or any other paste you used in school)? If not, Put it back on the heat for 20 - 30 seconds, swirling it as you do. Once it's got that gloss, and the milk has expanded a bit, it's stretched properly.
An electric burner is ideal for this, but there are also metal overlays for gas burners that just provide a hot surface.
At that point you can use a hand whisk, or any number of the frothing gadgets that you can find in any cooking store (brick-and-mortar or online).
You can, of course, just froth the milk, but what you're essentially doing is aerating it. That's fine, it's going to make a tasty drink, but it might fall short of your expectations if you don't stretch the milk as well. The process is what makes the milk and the coffee fall in love and give birth to a latte.
The steam wand on espresso machines heats the milk and adds the air, giving you a lot more control over the process. However, this will work almost as well.
Tips:
- Don't ever let the milk get close to simmering. It'll burn very quickly.
- Use the handle if you have to put the pitcher back on the heat (as naturally, you took it off because it was approaching too hot to hold)
- You want to get the milk in the coffee as soon as possible once it has that gloss. Don't let it cool down too much while frothing (work fast)
- It's going to take some practice
Total for this would be about $15 for the stainless steel pitcher, $10 - $15 for the frother, so all total around 30 bucks. You could use a sauce pan, but it's much harder to gauge how hot the milk is if you do, and much harder to control when you pour.