What software are you using to encode? Are you using two pass, or one pass? Do you have any other processes running? skratakh is correct in saying it isn't *that* fast when compared to more recent processors, and as Mike says, encoding is the most CPU intensive process.
The only way you'll get encoding time down significantly would be switching to a one pass encode, upgrading your PC, getting a hardware encoder or using alternative software (insignificant).
this is an
old test, but shows the differences in encoding speeds:
http://www.videohelp.com/comparison.htm (and for that matter quality).