Remove environment from rendering behind glass

Hello everyone,
I’m trying to get into motion tracking and always run into this issue:
If I use a glass material, the transparency reveals the environment texture instead of the background movie clip.
Is there a way I can isolate the transparency portions of the glass so that I can tell the compositor to change those portions of transparency into the background movie clip instead of the environment?

I will attach an image.
If you look at the glass windows, there transparency doesn’t show the background of the movie, but rather my environment texture.
Thank you in advance


!
Jesse