Never colocated, but did rent baremetal from OVH back when they didn't have any KVM and all you could do is wipe/reinstall, reboot and boot into a 2-3 releases old Debian recovery.

Definitely seconding the KVM remote access part: you really, really want that, or at least some way to hard reset your server if it crashes. I can't stress this enough. Even if you think you'll never need it, you never know when you'll have a kernel panic or need to do some boot troubleshooting, even just to run fsck. It's absolutely nerve-wracking to reboot a server you don't have any way to access other than SSH and looking at that ping window for 2-5 minutes while the thing boots back up and wondering it it will come back online or not.

If you don't have IPMI and can't have some sort of KVM for your server, I highly recommend having at least a PiKVM or something in there to be able to do remote troubleshooting. Ideally I also recommend (if no IPMI) setting up some sort of preboot environment you know will reliably boot (maybe something entirely in initramfs) that will boot up, get network and listen for SSH for a couple minutes before chainloading back into the main OS so that you can at least turn off firewall/reset network to known good. Anything that will give you remote access independently of your main OS.

At least I had access to the recovery environment from OVH, but even then, that thing took a full boot cycle to boot up + some more time for them to deliver the credentials by email (that better not be hosted on that box itself), change a config file, reboot again. Legit 10-15 minutes between each attempt and little to no way of knowing what happens until you boot the recovery again. It was horrifying, can't recommend.

IPMI saved my ass a few times and I'm never getting another box without it.