Here's how to fix this.
Add a key “ExtendedLocale” to the registry in the NLS folder. It can be empty.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\ExtendedLocale
After talking with a Microsoft Engineer this had to do with an automatic language pack removal. It was deleting that registry key for some reason. There is no way to stop the automatic language removal.