Low level software design is one of my regular work, most of the time it turns out to be really interesting; so much so that I fail to sense environmental changes around me :). Most of the decisions in such designs, confined to refactoring, are upfront.
There were times when these decisions used to drive me nuts (for example, I used to extend framework at larger granularity, failed to identify framework extensions at right places etc.), with time (and few refactoring book skims) I learned how to simplify it. Some of them are separating the concerns of mammoth classes into few "aware" type of interfaces (Spring style), distribute methods, extract stateless routines, derive common abstract base class etc. These decisions are quite easy to tackle with upfront designs.
Refactoring is wonderful, refactoring to patterns is even more interesting. Recently, Ketan identified design smells in his GUI testing framework for SWT. I did a test drive into the initial framework to understand and tried the same upfront approach to make it smell free, I failed; but I understood the potential pain points. Then, We brainstormed and discussed the design on white board, espied the smells; asking questions, suggesting and rejecting alternatives designs. That cleared few confusing points and we came with a good enough, smell free design (Ketan, all we did was shifting and distributing the bottleneck somewhere else :), as it can't be eliminated - "SWT"- I wonder why SWT is not very cleanly designed, performance?).
The lessons learned? You can't refactor well if you're in confusion (requirements or other), refactoring in this case drags you away, and Upfront designs takes extensive experience on variety of problem domains.