Skip to content

chore: standardise testing strategies for AI Tasks - #22021

Closed
SasSwart wants to merge 8 commits into
mainfrom
jjs/internal-1262-tests
Closed

SasSwart wants to merge 8 commits into
mainfrom
jjs/internal-1262-tests

Conversation

@SasSwart

@SasSwart SasSwart commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

closes: coder/internal#1362

coderd/aitasks_test.go contains a few different testing strategies. Some of the tests are also overly specific and should be generalized and then moved to the appropriate abstraction layer.

This PR standardises our testing strategy for coderd/aitasks_test.go.

Feedback to address:

#21889 (comment)

Most, perhaps all, task endpoints get a workspace at some point. We have endpoint specific testing to ensure that if a workspace cannot be gotten, we fail correctly. Feedback from reviewers is that we'd like to enforce this failure mode for all task endpoints instead of testing it per endpoint. This is to avoid the risk of inconsistent behaviour between different endpoints.

Furthermore, Task Pause and Resume endpoints use a wrapper that induces the failure by overriding the method that we need to fail. Elsewhere in the tests, dbgen and dbfake are used to induce failures using real state in a database.

The benefits to this are:

  • we know which states in the database induce failures by enumerating them
  • we avoid the burden of testing failures that are impossible right now

The drawbacks are:

  • tests are more tightly coupled to the database layer, which introduces friction to future changes in this space
  • tests assume the current state of the schema. As the codebase evolves, new failure modes will become possible that these tests will not account for.

Action items

  • Write a test to enumerate all task endpoints and ensure they fail correctly if a workspace cannot be retrieved.
  • Remove the aiTaskStoreWrapper and use dbgen/dbfake instead

#21889 (comment)

The original issue stated requirements related to authorization. These requirements were explicitly tested at the endpoint level, but those tests are deemed too specific and too granular. Per reviewer's request, these tests should be generalised and rather done at the rbac/dbauthz layer. At the endpoint level, it is sufficient to test using a second member user and ensure that the response is correct.

Action items

  • Ensure that all task, workspace and workspace build resources used by the AI tasks feature have sufficient RBAC enforced and tested at the dbauthz layer.
  • Simplify authz tests at the endpoint layer to only check the response type using a second member user.

#21889 (comment)

Similar to the feedback above, reviewers request that we generalize testing for when resources cannot be found across all AI Task endpoints and ensure that the 404 response is correct.

✅ #21948 (comment)

Define variables closer to first use.

the endpoint is now functional. however, there are a few non-functional changes to still make. 1. The pause and resume endpoint are nearly identical. It might be good to deduplicate them. 2. The authz tests need to be generalized
@github-actions github-actions Bot added the stale This issue is like stale bread. label Feb 19, 2026
@SasSwart SasSwart mentioned this pull request Feb 20, 2026
5 tasks done
@SasSwart SasSwart removed the stale This issue is like stale bread. label Feb 20, 2026
@github-actions github-actions Bot added the stale This issue is like stale bread. label Feb 28, 2026
@github-actions github-actions Bot closed this Mar 3, 2026
@github-actions
github-actions Bot deleted the jjs/internal-1262-tests branch August 12, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tasks: Standardise testing for AI Tasks API testing

1 participant