You have my permission, as if you needed it, to wear something more casual than black tie on the holiday that is the first day of the new year.
Public appearances on days like today are the proper domain of the casual suit, by which I mean something patterned like the one worn by Charlie Watts in the photo, or rumpled like anything linen, or both. Whether woolen, linen or worsted, the closet of every suit-wearing man should have at least one casual suit for cool weather and one for warm (two per season is preferable if you are going to wear them with any frequency). And men who do not wear suits in their work lives may have them for social occasions.
Choosing casual suits requires a man to walk the border between city and country. Pinstripes, or any pattern automatically acceptable in the office, do not meet the criteria and neither do loud tweeds such as the ones you might wear to go shooting. Choose instead a houndstooth or glen check where the check is too large for business, or one of the more conservative tweeds such as a dark green herringbone.
Casual suits need cost no more per wear than the rest of a man's wardrobe. The day after is far from the only time the they come into their own. Casual suits are ideal for travel, assuming you are not a man who chooses fleece for the first class cabin of an aircraft or the Eurostar (one of my fonder pre-marital memories is of being picked up by a flight attendent who was captured by my gray herringbone).
The best argument for casual suits to my mind is that they are so rarely seen. Everyone has odd jackets. To stand out without shouting on the day after, wear a casual suit.