A board can be deleted when a capture is created and the user can restore the board within 30 days. If this happens, the Activity Log indicates a board_state_changed event, but no item events since the items were not updated. However, we never backfilled the items, so we need to fetch all items for this board as long as the state change isn't from active or archived to deleted.

The Monday.com API docs do not document any of the Activity Log events. So I have had to discover this through testing and use of their API. This TODO may need to be removed or just left as a note, since right now I am unsure a straight forward way to know a board was duplicated. In testing, when I duplicate a board with all items it shows events that don't mention duplication or anything related. My only idea right now is to maybe key of some create type event and determine if a board was just created and then just trying to fetch it's items in case it was a duplicated board+items.

The while True: loop at first glance may be confusing. Note, though, the get_board_cursor() can only be called after the items generator is processed since under the hood this is using the IncrementalJsonProcessor and remainder processing to pull out the necessary information after all items are streamed from the response first.

How does this behave when an old board is updated before the items backfill has fetched items for that board? I'm thinking of a scenario like:

1. cutoff is 2025-06-01T12:00:00Z
2. The fetch_boards_minimal returns:

[
  {"id": 3, "created_at": "2025-06-01T11:59:00Z", "updated_at": "2025-06-01T11:59:00Z"},
  {"id": 2, "created_at": "2025-06-01T11:58:00Z", "updated_at": "2025-06-01T11:58:00Z"},
  {"id": 1, "created_at": "2025-06-01T11:57:00Z", "updated_at": "2025-06-01T11:57:00Z"},
]

3. The fetch_items_page backfills board 3, then returns and checkpoints.
4. On the next fetch_items_page invocation, board 1 has been updated and fetch_boards_minimal returns:

[
  {"id": 3, "created_at": "2025-06-01T11:59:00Z", "updated_at": "2025-06-01T11:59:00Z"},
  {"id": 2, "created_at": "2025-06-01T11:58:00Z", "updated_at": "2025-06-01T11:58:00Z"},
  {"id": 1, "created_at": "2025-06-01T11:57:00Z", "updated_at": "2025-06-01T12:01:00Z"},
]

It looks like with this filtering logic, board 1's items would be missed in the backfill. Is that a valid concern?

If so, I'd suggest using the board's created_at instead of updated_at for the backfill cursor to avoid this kind of race condition.

We'd also need to process boards in descending order of their created_at to make sure we don't skip a board. Since Monday's API returns boards in descending order of their created_at, I suspect this is already handled, but it would be good to confirm the item_iterator.get_items_from_boards doesn't shuffle the board ids around somehow.

Hmm. I am thinking about this still, but a couple things to note about this that are challenging:

• API only allows a query that filters by created_at in descending order
• API response does NOT return created_at only updated_at

Wow, I didn't realize Monday doesn't let us query created_at for boards. That's frustrating.

Other ideas off the top of my head that might work:

• Keep the logic you have with updated_at in fetch_items_page, but in fetch_items_changes yield all of a board's items whenever we see a board is updated. (I think there's a reason this wouldn't work from what you've mentioned before, but I don't remember exactly what that is. Maybe something about some item updates not updating the parent board?).
• Instead of using updated_at as the page cursor, store the ids of all boards as the page cursor. Then fetch the items for a single board in fetch_items_page, remove that board id from the page cursor, then yield the updated list of board ids.

There are likely other approaches too, but those were ones I thought of right now.

The first approach "could" work but the fetching/pagination of board items is costly and pretty slow in general. Items being updated/added to update the parent board updated_at too, but the parent board updated_at can be changed for other reasons too, so I would expect a lot of false positives.

I like the second approach. Is it safe to say there is not currently a limit on the PageCursor string length/format built into the CDK? I will look myself too, but was just thinking, if we have 100k+ boards that would be a large list of IDs for a cursor that we'd parse each time since the PageCursor can only be str | int | None and we'd have to have it as comma separated string for simplicity. I could probably make it an integer and use bit manipulation to extract the next board or set of boards for a page, but will keep it simple (CSV based)

I'm not aware of limits we impose on PageCursor about the string length or format. There are checks we perform on subsequent LogCursors (are they the same type & are they increasing?), but there are no such checks on PageCursor so I think the CDK would support a comma delimited string of board ids for a PageCursor.

You're right it could be a pretty large string if there are a lot of boards, and I agree with keeping it simple. It's not the ideal approach, but it feels like we have to get creative with the API restrictions Monday.com enforces.

It may be worthwhile getting @williamhbaker's thoughts & if he sees a better approach.

The thing to watch out for with using a really long string as a cursor like that is that it will be the "value" in a JSON key/value map...every time the connector commits a checkpoint, this gets written to the recovery log, so if it is committing checkpoints with any kind of high frequency during the backfill then every time that huge string needs to be written to the recovery log, and it seems like it would cause very large recovery logs which are problematic.

Ideally such a checkpoint would be structured such that each board ID was the key to a map (which a dummy boolean value or somethign), and you would use JSON merge patches to clear out IDs that are completed. A connector checkpoint can be very large itself without much downside, but the checkpoint updates need to be kept small and applied with JSON merge patches to manage recovery log size.

I will address this in a separate PR since it is CDK level changes and needs more testing. I will have a separate commit though addressing other items.

nit: It doesn't look like get_cursors is used anywhere. Can it be removed?

nit: It doesn't look like _items_generator_started is referenced anywhere, so can it be removed?

nit: oldest_board_updated_at is already a public property, so I'd suggest accessing it directly instead of having a get_oldest_board_updated_at getter.