TLDR: Keep retrying the configuration when it fails during centralized upgrades, and maybe you'll be successful.
Currently upgrading from 2020.2.1 HF2 to 2020.2.4, and it's been fairly rocky. I'm upgrading via the centralized upgrade process using the offline method. So far I've had to retry the upgrade on the primary Orion server no less than 7 times as it's failed for numerous reasons. After all the retries, the Primary server is finally complete and the additional polling engines are going through the process now. Will update if I run into other issues.
Modules Running:
NPM
NCM
NTA
Edit: Only had to retry the configuration on one additional polling engine one time. Looks like I'm completely on 2020.2.4 now!