At the moment, AAE does not handle objects that expire. As such, even though the data has gone, the entry in the AAE hashtree remains until the hashtree is cleared and rebuilt. Should you have a lot of items that expire, this can cause AAE clearing to take a very long period of time.

Quick example:
I store user sessions in one backend and they expire after an hour. AAE runs once monthly.

Assuming that AAE just completed:
After 1 hour, I have 1 hour's worth of data and 1 hour's worth of AAE data. All good.
After 2 hours, I have 1 hours' worth of data and 2 hours' worth of AAE data. Ok but not ideal.
After 3 hours, I have 1 hours' worth of data and 3 hours' worth of AAE data. Ok but not ideal.
After 24 hours, I have 1 hours' worth of data and 24 hours' worth of AAE data. Not very happy.
After 48 hours, I have 1 hours' worth of data and 48 hours' worth of AAE data. Annoyed now.
After 30 days, I have 1 hours' worth of data and 720 hours' worth of AAE data. Somewhat insane.

Now AAE gets to clear the hashtree but during the hashtree clearing process, should more than 90 hashtree tokens be used, the node becomes locked until clearing completes i.e. incoming writes have to wait and start to time out.

As hashtree clearing deletes one item at a time from the hashtree, the above hashtree bloat becomes a significant time waster. Even assuming an average of only 500 sessions per hour that is 360,000 records to be cleared where there should only be 500. Estimate 1 ms to clear each item from the hashtree and we observe 100th percentile node put fsms taking in the region of 5-6 minutes to complete in the worst cases.

To address the above, I have two suggestions:

1. Adjust expiry to be a regular delete i.e. when called, it deletes the entry from the AAE hashtree at the same time it deletes the data to be expired.
2. Adjust the way that hashtree clearing works so that it deletes all data in one go ala rm -rf <folder> style rather than the current iteration over each item.