This proposal provides an implementation of TTL (#746) assuming the application code inserts documents with timestamps. It can be combined with #5813 to avoid that requirement on the application code.

The idea is to trigger deletions whenever the index value of a document passes the current time.

Configuration API

Proposal for an API for configuring expiration on a given table:

• tbl.setExpiration(indexName[, replaceFunction]) sets expiration based on the given index indexName. Optionally, a custom function can be provided that allows writes other than deletion when a document expires (e.g. setting a flag on the document). The default is replaceFunction = function(oldVal) { return null; }.
• tbl.setExpiration(null) disables expiration
• tbl.getExpiration() retrieves the current expiration configuration

The setExpiration term requires "config" permissions on tbl.

Notes

Care must be taken when specifying a non-default replaceFunction. If the function fails to lower or remove the value of the index indexName for the document, expiration will not work (though we will catch this case, see below).

Implementation

Runs on each primary replica of the table:

1. Run tbl.between(r.minval, r.now, {index: indexName}).replace(replaceFunction) limited to the shard's range
2. Fetch next = tbl.min({index: indexName}), evaluate nextTS = indexFunc(next), limited to the shard's range
3. If nextTS is not a datetime, log an error, sleep indefinitely.
   (Detail to detect bad replaceFunction: if next has the same primary key as the next from the previous iteration, print an error and sleep indefinitely as well.)
   Otherwise, sleep for nextTS - r.now.
   In either case, the sleep gets interrupted if any write on the primary's shard range
   introduces an index value that's smaller than nextTS.
4. Go to step 1