Yes the base impulse is there. The base impulse to be attracted to people other than your spouse is there too. But as humans you can choose to rise above animal instincts and think about the consequences of your actions. You choose not to punch coworkers in the face when you are angry or hump the hot surfer dude changing out of his wetsuit. That’s what separates us from the apes. I’m sure the happily married people here who haven’t cheated on their spouses can agree that they have been sexually attracted to someone other than their spouse at some point and chose not to act on it. Why do you think we watch Brad Pitt/Angelina Jolie movies?