Think carefully for a moment; do atheists who get a pitying expression use words that invite it?
Maybe 25 years ago I told a very bright woman that I seem to attract passive women. Without a moment's hesitation she asked me if something in my manner invites such women.
I was ready to see it (no longer needed to deny it) and I saw it. Happily I was doing volunteer work that gave me opportunities to practice new ways to relate.
I must have succeeded because right now I don't remember what I changed. I also no longer have that complaint; I enjoy occasional verbal combat with assertive women.