Short

TL;DR: Read the man page man pveproxy

Put the following in /etc/default/pveproxy (it needs to be created if not already existing).

LISTEN_IP="127.0.0.1"

Then restart pveproxy.service.

That's it.

Oh it's also mentioned in the manpage but it's worth reiterating, tweaking this setting after setting up a cluster is asking for an issue, the cluster normally communicates directly on 8006.

This could break something pretty major so uh, not my fault.