That's how I roll, too. Don't want anything bulky in my pocket and don't want anybody watching me fumble through my wallet at the vendor stands, etc. I usually carry my ATM card, as well, however I've never had a taxi driver turn me down when I have run out of cash. They will be happy to wait outside your hotel while you run to your room to get more money.
But yeah, my wallet stays in the safe with my credit cards and ID.