He claimed in his autobiography to have much less, but according to Patrick Grant of Norton & Sons the late Duke of Windsor's wardrobe was fairly large at the time of his death:
-55 lounge suits
-15 evening suits
-3 formal day suits
-more than 100 pairs of shoes
Now far be it from me to say that any contemporary man has too much or too little but once he exceeds twenty or so suits, a dinner jacket or two and a pair of shoes for each jacket his wardrobe may properly be classified as adequate to any likely occasion. Not that he is likely to win any awards at that level - the founder of Revlon died with more than two hundred navy blue suits after all - but any more and he has entered the territory of the clothing hobbyist whether he cares to admit it or not.
Personally, I was glad to see that the Duke had more than he let on about. It gave me license to think about shoes again.