Suppose I am having a conversation with a person who is unfamiliar with Jewish tradition, and the person casually attempts to pronounce the Shem Hameforash according to its letters, as some secular historians do. Should I utter a response of some kind, such as ברוך הוא וברוך שמו οr ברוך שם כבוד? Does it matter what the attempt sounded like?
It depends on if the person believes in God in the way Jews do: As a single entity with no shituf (so this excludes most, but not all, Christians).
So it depends on the intent of the speaker when he said the name.
You have a separate problem that the person is unlikely to have pronounced the name correctly. So it is probably not God's name anyway.