Alfred's, about a km from town, is where we always stay. The rooms are basic w/o TV or phone, but we love it, and an oceanfront room is only $60/night.
Not to mention the best rum punch on the beach! Does get loud though on the nights they have live music.