No! It's a RFD urban legend craftily promulgated by tire makers and dealers who want to move more product.
If the winter tires are brand new and have something like Blizzak technology with a relatively soft topmost layer, then yes, they will wear down quicker on dry pavement in warm weather.
But for most winter tires that have already done a few kms., it's not going to be the rate of wear that is popularly supposed.
And even in the worst case, you arrive home with what is effectively now a perfectly good set of all-seasons that provided you a safety margin had snow been an issue in some parts of the trip.