If black tailored clothing is generally misapplied, white, like the jacket worn in the photo by the late Duke of Windsor, is also a challenge. White is as difficult as black to combine with many complexions, and loses some of its attractiveness as it acquires a yellow cast over time from oil in the wool (fortunately, cotton shirtings do not have this problem). That failing is of course why most of what we call white is actually cream, a shade that's more flattering to wear and less likely to age badly.
White also shows every bit of dirt, making it about the most expensive color a man can wear. If he's not having his white clothes cleaned, he's disposing of them prematurely because of some stubborn spots that refuse to come out. I know my own cream linen trousers last less than half as long as do the brown versions next to them in the closet.
Instead of white, the prudent man will choose lightweight black or midnight blue mohair for warm semi-formal evenings. Mohair wears cooler, and it will be out of the closet much more often than a white jacket that is, after all, only appropriate for outdoor and seaside occasions in warm weather.
This is not to say that there is anything wrong with white or cream for day wear, mind you, particularly next to tanned skin. Cream flannel, linen or gabardine trousers are as elegant as anything in a man's wardrobe, cream cashmere sweaters are delightful, and an odd jacket of cream silk herringbone is particularly good looking. But all these things are relatively expensive to begin with, and their short lives make the cost even greater.
White is a challenge.