I see Next.js more as a build system than a framework / library. Of course it adds complexity, but in a comment above you say that "CRA does that too". You're not introducing any more or less complexity than CRA.
There seems to be a misconception that you need to go all-in on utilities when using a framework like Tailwind.
If you have 500 buttons using the exact same set of classes, you should of had a button class in the first place. Almost all my projects have their own button and form input classes for this exact reason!
Where utility-based CSS frameworks really start to shine is the other 75% of your codebase, where you're inventing clever class names, only to end up using them once.
This is also why Tailwind calls itself a utility-FIRST framework. Build things with utilities, and extract components to classes when the need arises. Avoiding early abstractions provides a lot of value (in any programming language for that matter). Not having to invent a name for every single piece of UI also saves a ton of headaches down the road.