As I was moving my Windows 10 installation from an old borken HDD to a new, fresh SSD I ended up just using Microsofts super-easy Media Creation Tool to create a bootable USB stick. I booted it up and created a new installation on my new SSD after formatting it. All of this went great and I was happy as a hamster.
Then, when I rebooted, I got this horrible screen:
I had no idea why this happened and I got it on every boot, so I did some frantic googling and I found in various suspicious forums that the solution was to do this:
1. Open the windows run dialog (Win + R)
If you save that and reboot (after you have removed the other Windows 10 entries your boot section should look like mine in the above image, minus the enormous red arrow).
Finished, and the problem is gone!