As title states,

I have a handheld camera which is the only moving object here, the object i wanna "mask"? and have it only be in color in the scene the rest in BW.
Thing is the handheld camera moves so the house changes angles and there are some colors in the BW that are the same as the object.

Any tips?
Point me to a course or video and ill be a happy camper!