yup. A good intellectual test would be asking whether people would draw the same conclusion on intent about the gesture if:

A) it were sexual coming from a person with a history of sexually harassing comments who provides a sexual joke for explanation B) it were a direct insult (like middle finger) from a person with a history of verbally abusive comments, who provides a fuckyou joke for explanation.

People usually have very little difficulty judging the intent in those situations.