And in this instance, for a visitor to the island to say "no thanks" will not be taken as a slight. There is no need to over think such a simple concept. If you dont want to get in that particular red plate for any reason - that is your right! There is no need to explain anything. A simple "no thanks" will suffice in every single case.
And there is never any reason to get into a white plate illegal taxi. A "no thanks" will work just fine - no need to get into any detailed explanation with the driver - just say "no thanks". If they get angry at you, so what? They are an illegal taxi and you are not under obligation to use their services. Chances are anything you would say would make them angry - so a simple "no thanks" will work just as well as anything else. As the saying goes, you cant fix stupid.