666 the Number of the Beast??

This happened a year ago, but I was reminded of it by the same guy who said it to me then. A guy that I work with saw my skull and crossbones tattoo and the first thing out of his mouth was not "hey nice tattoo" or "wow you have a tattoo". The first thing that could come to his head was, "Do you worship the devil?" Now granted I had had a conversation with him earlier about how I had used to be in a hardcore band (I also defined what hardcore music was), but is it right for him to assume I "worship the devil" if all that I have is a skull and crossbones tattoo? Am I that menacing looking? Maybe I need to cut back on my blood lattes...