Two times I had mine taken when trying to bring it home. Customs said no. Maybe just my bad luck?
I don't think its bad luck, because I saw a couple on my last trip that had their hat thrown in the trash. The officer said it was not to leave the island.