Putting an xz Backdoor Payload in a Valid RSA Key

Last week, a backdoor was discovered in xz-utils. The backdoor processes commands sent using RSA public keys as a covert channel. In order to prevent anyone else from using the backdoor, the threat actor implemented a cryptographic signature check on the payload.

I have seen a number of people claim that this would necessarily result in an obviously invalid RSA public key, or at least one with no corresponding private key.

This is incorrect, and someone nerd sniped me into proving it.