Improve agent test coverage #614
Conversation
Since the inclusion of "gpt-5" and "gpt-4.1" conditions in the model family that generate different prompts (see agentPrompt.tsx and agentInstructions.tsx), this commit does tests for such families, improving coverage and updating snapshots.
- Modified agentPrompt.spec.tsx to run tests for different model families ("default", "gpt-4.1", "gpt-5") using a loop.
- Updated MockEndpoint instantiation across various test files to accept an optional family parameter.
- Adjusted agentPromptToString function to handle family-specific endpoint creation.
- Ensured all relevant tests are updated to reflect changes in endpoint creation and model family handling.
- Cleaned up test cases for clarity and consistency.
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
Sure, thanks for being so passionate about unit tests 😄
fyi @bhavyaus
😄 Yep. In fact, I’d like to try refactoring the agent prompts to move all the “GPT-4.1” and “GPT-5” family-specific additions— and potentially any future ones—out of the generic prompt files. If more family-specific prompts are added, the agent-prompt system would become difficult to maintain. But first, I need to make sure that such a refactor wouldn’t cause any issues, so the first step is to ensure that the GPT-5 family prompts are generated during tests! |
|
Sure. I agree that the dynamic inclusions make our prompts a bit hard to follow. I'm open to refactoring a bit to make it easier, even if it means a little duplication between prompts per family. If you're going to spend a lot of time on it, I recommend opening an issue with a description of what you want to do so that I and @bhavyaus see it and can coordinate, I just wouldn't want you to waste time on anything. |
4 participants
Since the inclusion of "gpt-5" and "gpt-4.1" conditions in the model family that generate different prompts (see agentPrompt.tsx and agentInstructions.tsx), this commit does tests for such families, improving coverage and updating snapshots.