It seems that we don't actually need arc-swap for the memo table, because our deleted_entries queue takes care of deferring reclamation until there are no live borrows of a given memo. arc-swap does a lot of extra bookkeeping; replacing it with a manual AtomicPtr object speeds up red-knot's incremental performance by about 15%. This should also make it easier to test with loom/shuttle.

You'll notice that none of the safety requirements of the MemoTable functions actually change with this PR, so I believe it is sound. The one catch is that we have to avoid returning ManuallyDrop<Box<T>> and instead work with NonNull until the allocation is actually freed and we can assert uniqueness, as Box is a noalias type (at least under the current rules).

I also played around with using boxcar::Vec instead of the current lock, but the lock is actually relatively uncontended and fine-grained and it was hard to balance cold and incremental performance with a lock-free implementation.