GDALCreate has been doing the cleanup thoroughly since 3.2.0. I discovered this while testing the opener plugin from #2907. The opener gets called twice with mode=r. Once by _delete_dataset_if_exists() and once again by GDALCreate().