We stay at White Sands, very kid friendly. Our son is 7 and we traveled with him when he was four and again just last month. This was our 4th stay at White Sands and every visit there were kids from a few months old and up.
I also choose to bring my son as he is part of our family and a family vacation it is. We went without him once when he was 2 and I would never leave him again, wasn't the same without him. We had a gentleman say to my husband our first trip that he hated when kids were around him on the beach(very good thing I did not hear him say that)I think hubby was just shocked and walked away. The same guy later commented to him that our son was so well behaved he didn't know he was there.
Sorry if bringing my family on vacation ruins yours but it is what we do, like it or not.