We should expose real API to bring up a modal dialog and use it in this case:

/cc @jrieken

I can look into this / provide a PR if there are no objections.