I guess they have very sturdy concrete homes in there, and they resist the wind. The incoming water surge is probably the main problem there. A lot of homes in the Caribbean are built in reinforced concrete, very different from the States that usually built in Gypsum, wood and very fragile materials.
Of course wood is also a typical construction material in the tropics, but because we live in islands we have a different approach when a Huge hurricane is nearby: we never evacuate, because we have water in all directions, so usually government use concrete facilities to hold refugees, during the event. That way they minimize deaths and everyone is "secure" during the storm.
Other usual risk is rivers and landslides, specially if you live in mountains, that's a sure killer, and refusing to leave for a refuge it is also a killer.