That's how it goes for me too, I use my card all the time, at home and abroad. I have had to get it replaced twice an both times the CC company alerted ME to the fraud.I filed a claim and my money was immediately returned to my account.
Honestly I don't stress about using it. I have zero liability for fraud on it.
However, I do not use my debit card as a credit card, only my credit card. Getting money back in an account is a bit more of a pain than simply having charges removed from a not-yet-paid bill.