Code that has been in use for any length of time tends to rot. Design decisions that made a heck of a lot of sense five or ten years ago, when machines were less powerful and memory more expensive, seem silly -- perhaps wrong. As revision after revision is piled on, it tends to make a mess. Add to this the hands of a hundred programmers -- each one knowing the one true way to write code -- and a lot of code tends to become a hopeless jumble.
Worse, that hopeless jumble is tested and works.
As Joel Spolsky explained, modifying existing code has two dangers. First, code is hard to read, and that hopeless jumble you're trying to enhance already does things that you won't understand without a lot of work. Second, the more you alter the code, the more you'll have to retest to ensure you haven't broken the existing functionality.
Now, if you are lucky enough to have a full suite of unit tests, then you can let yourself go wild and refactor with the assurance that the unit tests will catch any errors. But this is old, revised code. Even if you have unit tests, they probably are out of date or perhaps they no longer work.
One thing that I remember from Boy Scouts -- you have to have the right attitude toward camping. Ideally, you should leave a campsite without any evidence that you had been there. That's probably not possible. But our scoutmasters had a simple for us to follow: always leave the campsite cleaner than when you arrived.
This rule applies equally well to code. No matter how badly some bit of code has been abused in the past, you can always have it leave your hands in better shape than when it arrived. If the code needs comments, add them. As you figure out how something works or some obscure requirement that's buried in the code -- note it for future programers.
If the code has a jumble of if-statements, and you can restructure it to be more clear, do so. If you find dead code, eliminate it. Just be ready to test it fully before you leave it for the next developer. No matter how much you temper your changes, you can always do something to improve the code for the next guy.
And if everyone did this, we'd have fewer hopeless jumbles.