Admin Area -> Users -> Options:
Allow users to delete account: enable/disable

(enable will show the delete button)
(disable will hide the delete button)