don't know if it's me misunderstanding how it's supposed to work, but "Mask image by brightness" doesn't behave as I'd expect. My suspect is that "blue" and "green" parameters into "Threshold types" are inverted. Check the following example:
Ways to obtain background substitution changed a lot since a few years ago. I did it with background subtraction some years ago - you need to freeze a frame of the background without anyone in, then you do a pixel subtraction frame by frame. Nowadays best results are obtained through machine learning techniques. This works pretty well for a portrait pose like the one in your mask. One good tool that does it is Zoom - check its "background removal" function. Others are Snap Camera (and its tool Lens Studio that allows you to build the desired effect), and ChromaCam. All these allows you to put a custom image or video as background. At worse, you can put a green background, and do a greenscreen-like post step in Vuo. I think that some experimentation with these, joined with some Syphon, could work.