You may be able to also boost the voices a little to compensate.
The unfortunate thing with introduced noise is unlike line hum, it often occupies a wide spectrum of frequencies making it very difficult to filter out without affecting the audio you want to keep.