If I saw a poster with "atheist band" on it, I would interpret that as "anti religion band". Bands call themselves "(religion) bands" to distinguish themselves from the default position of just playing music with no regard to any religion. From the look of your video, my definition of "atheist band" seems appropriate.
Same thing with comedians. If someone calls themself an atheist comedian, I expect to hear anti religion jokes included, as opposed to a normal comedian who might do jokes about anything. Similarly a christian comedian should have christian jokes included if they use that label.
However, I don't know where you live and the general culture may not reflect the view I have written above, so you would have to decide based on your potential audience's views.
Freaky indeed. People are here tend towards kind and thoughtful. If I didn't know better, I'd think they all were some upstanding Christians. ;)
As for the band, I'd go for it! Like K said, Christian bands are cheesy because Christianity is. You'll be fine! Besides, I'd love to have an atheist CD to pull out and play when someone else was bothering me with their religious music!