Guide used:
https://stellarlab.net/azure-automation-with-exchange-online-part-1
App Registration
The exchange runbooks in azure use the "ExO PowerShell CBA" app registration with certificate based authentication to connect to exchange management shell.

Certificate
ExO Powershell CBA has the certificate stored on it's object:

To create/"renew" certificate
New-SelfSignedCertificate -Subject "CN=$Name" -KeyLength 2048 -KeyUsageProperty All -KeyAlgorithm 'RSA' -HashAlgorithm 'SHA256' -Provider "Microsoft Enhanced RSA and AES Cryptographic Provider" -NotAfter (Get-Date).AddYears(100)
Create certificate on any machine, export including the private key, additionally export without the private key:

Make note of the private key in password manager and store in CRST IT Support - Documents\01_Central Store\2_Documentation\0_Central & 365 Network Stuff\Automation Certificate
Upload the .cer to the App Registration mentioned above.
Upload the .pfx to the Automation account

Finally, update the cert name in the automation script:
