As someone else in management I would take the higher morale boost I have from my completely remote engineering team to some “good conversations” and “interesting ideas” that may pop up from time to time. It’s not their job to come up with that shit. It’s either mine or products and they are definitely able to do it organically if you facilitate casual working sessions that promote conversation over productivity.

Edit: also if you have people who can’t manage their work without you stating over their shoulder that’s a failure on your part not theirs. Yes some people just aren’t cut out for some jobs but if the difference is being remote or in the office that is totally on you.