What's the worst that can happen? Well, that is rather open ended... lol.
For this answer I am assuming that you have clear legal title to the land and that you are talking about preventable "worsts" as opposed to acts of God.
Land and/or a house left alone can lend itself to unwanted "squatters", both animal and human. If a neighbors' goats, pigs, etc. have made your land their home, then you would have to ask the neighbor to get them out. If the animals do not belong to anyone else, depending on what animals that they are you may need to contact the proper authorities to get them removed. Some wildlife are protected species.
As for human habitation, you can also contact the proper authorities to have them ordered off the property. Squatting is becoming less of an issue for landowners, but squatters still have some rights depending on the circumstances.
On a more personal issue for those who live here, leaving any property unattended for any length of time attracts some issues in the Caribbean. Buildings, rather like cars, tend to die a quicker death when not being used. With the humid sea spray infused climate, rust, mold and general decay happens quickly.
Having someone tend to the property by hire, renting or free would all help to minimize the above issues.