I was stripping out a Windows 10 VM and gpgfe was not needed on it. I am using Windows 10 United Kingdom (en-gb) and the uninstall dialog boxes have only pictographic, I think Chinese, characters in the text. Seems to fuction normally as there do not seem to be any uninstall options. No non-romanse language or IME is installed. I have tested this on a physical install and it is the same.

On the install, it may be worth considering a forced always on top for the install wizard dialogs.