You can do it in AE or even photoshop. Basically you 'hand paint' the effect into each and every frame. It's mind-numbingly tedious. To quote a colleague of mine, 'that's why we pay the guys in China to do it.' I've only bothered to do it the once because in my experience it's cheaper in time and money to completely reshoot the piece 'properly' than to rotoscope it. YMMV.