It would be handy if this warning included a link to the media library to add alt text to the file:

There may be a more elegant way to do this, but you could add a class to <img> tags containing the image ID using the hook wp_get_attachment_image_attributes:

function add_attachment_id_on_images($attr, $attachment) {
	if (!strpos($attr["class"], "wp-image-" . $attachment->ID)) {
		$attr["class"] .= " wp-image-" . $attachment->ID;
	}
	return $attr;
}
add_filter("wp_get_attachment_image_attributes", "add_attachment_id_on_images", 10, 2);

Then you could extract that ID using some Javascript and use it to link straight to the media library by just slotting the ID into the URL: /wp-admin/post.php?post=<id>&action=edit.

This would not be 100% bulletproof since images in the block editor only pull alt attributes out of the media library when initially added to a post, but anything in the block editor is already easy to access using the edit link in the toolbar, and having this shortcut would make adding alt attributes to other sources like featured images just as quick.