Def the alternator brushes are on their way out. Its not providing enough charge to recharge the battery and run the lights in your car at the same time (yes, your battery could also be suffering). You shouldn't need to replace the entire alternator, just the brushes only (which is that little black box part at the back of the alternator, which your loom plugs into.
No idea on price, but alot cheaper than another alternator.