You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue is for collecting feedback on the issue triage bot; areas where it could improve, individual issues that it didn't triage well, possible prompt improvements, ...
Please feel free to create new comments here for feedback.
Feedback: The bot triaged this macro related issue into area-test, likely because the issue discussed failing tests. As this is a common scenario, area-test should be weighted down somehow or have even more explicit prompting around preferring functional area.
Similarly to the previous comment, for transferred issues (like dart-lang/sdk#56072), we should at least feed in the first and last comment if not all the comments.
We may also consider introducing a slight delay in triaging issues (2 min?). This would allow some time for manual triage.
We should auto-close Analyzer Feedback from IntelliJ issues; these almost universally do not contain information (are not customized over the default template that just sends version info).
Yeah, I agree that that summary is off; you were asking for an analyzer code assist and it summarized it as a language enhancement request. I don't know if that means we need better prompting here, and the model is already well-trained on the Dart language. Perhaps a better model? General incremental improvements in gemini's capabilities? We're using gemini-1.5-flash to classify now, I could try a dry run w/ just regular gemini 1.5.
Maybe a way to ask the bot for a review? Than you could have a little more context into the new review
Do you mean to interact with the bot a bit to improve the summary? I think it would be reasonable to re-prompt it based on responses to its summary / classification comment.
Exactly. Maybe a keyword (or even pinging it) to ask for it to review the OP and the keyword comment. I believe if that is to become a thing, it should probably always appear at the end of its comments, like: "If you disagree with me, do this to make it so that I review this issue again".
About the above comment, not sure if it has been addressed or what (maybe a tag-specific problem), but I just created a new issue dart-lang/sdk#56631. Got no email errors and the bot could place the tags normally.
We had a regression in the triage bot - it was briefly trying to assign too many labels and the label assignment part of the workflow was failing; I suspect that's what you were seeing.
But why does the fork gets the email? I see that I have not received any other issue so only issues I wrote but still don't understand why am I receiving the emails? Is this a bug on GitHub Actions or something?
But why does the fork gets the email? I see that I have not received any other issue so only issues I wrote
Good question! I'm not sure why you're being notified. Perhaps it's related to your fork or perhaps related to you being the author of the issue that's triggering the workflow.
Had an issue with a type-question label, but no area- label, where the bot added an area (correctly) and a second type- label.
It should proabably not add any label for which there is already a label with the same prefix.
I can generally say that area-language is more wrong than right. (There isn't much use for area-language in the SDK repo after the language repo was created, so it's unlikely that area-language will be correct.)
Activity
parlough commentedon Jun 14, 2024
Issue/comment: dart-lang/sdk#56002 (comment)
Feedback: The bot triaged this macro related issue into
area-test, likely because the issue discussed failing tests. As this is a common scenario,area-testshould be weighted down somehow or have even more explicit prompting around preferring functional area.natebosch commentedon Jun 17, 2024
dart-lang/sdk#56028 (comment) would benefit from getting more context than the first comment of the issue.
devoncarew commentedon Jun 24, 2024
Similarly to the previous comment, for transferred issues (like dart-lang/sdk#56072), we should at least feed in the first and last comment if not all the comments.
We may also consider introducing a slight delay in triaging issues (2 min?). This would allow some time for manual triage.
devoncarew commentedon Jun 24, 2024
We should auto-close
Analyzer Feedback from IntelliJissues; these almost universally do not contain information (are not customized over the default template that just sends version info).devoncarew commentedon Jun 24, 2024
See #274 for a few of the discussed improvements.
devoncarew commentedon Jul 5, 2024
When an issue comment contains an image, we should pass that image into the gemini API (along with the issue comment's text).
FMorschel commentedon Jul 8, 2024
Maybe pass the current docs to solve cases like: dart-lang/sdk#56180? I'm not really sure what to suggest but the summary is off.
devoncarew commentedon Jul 8, 2024
Yeah, I agree that that summary is off; you were asking for an analyzer code assist and it summarized it as a language enhancement request. I don't know if that means we need better prompting here, and the model is already well-trained on the Dart language. Perhaps a better model? General incremental improvements in gemini's capabilities? We're using gemini-1.5-flash to classify now, I could try a dry run w/ just regular gemini 1.5.
FMorschel commentedon Jul 11, 2024
Maybe a way to ask the bot for a review? Than you could have a little more context into the new review
devoncarew commentedon Jul 11, 2024
Do you mean to interact with the bot a bit to improve the summary? I think it would be reasonable to re-prompt it based on responses to its summary / classification comment.
FMorschel commentedon Jul 11, 2024
Exactly. Maybe a keyword (or even pinging it) to ask for it to review the OP and the keyword comment. I believe if that is to become a thing, it should probably always appear at the end of its comments, like: "If you disagree with me, do this to make it so that I review this issue again".
FMorschel commentedon Aug 29, 2024
Not sure what's going on here but I have the Dart SDK as a fork (intend on helping out soon). Both:
Have sent me an email with failed jobs and have not been able to place the tags correctly, only commenting.
Maybe there is a bug here? Not sure if this is the right place or if this should have been a new issue.
FMorschel commentedon Sep 2, 2024
About the above comment, not sure if it has been addressed or what (maybe a tag-specific problem), but I just created a new issue dart-lang/sdk#56631. Got no email errors and the bot could place the tags normally.
devoncarew commentedon Sep 3, 2024
We had a regression in the triage bot - it was briefly trying to assign too many labels and the label assignment part of the workflow was failing; I suspect that's what you were seeing.
FMorschel commentedon Sep 3, 2024
I see.
But why does the fork gets the email? I see that I have not received any other issue so only issues I wrote but still don't understand why am I receiving the emails? Is this a bug on GitHub Actions or something?
devoncarew commentedon Sep 3, 2024
Good question! I'm not sure why you're being notified. Perhaps it's related to your fork or perhaps related to you being the author of the issue that's triggering the workflow.
lrhn commentedon Oct 22, 2024
Had an issue with a
type-questionlabel, but noarea-label, where the bot added an area (correctly) and a secondtype-label.It should proabably not add any label for which there is already a label with the same prefix.
ignore redundant 'type-' labels (#309)
devoncarew commentedon Nov 6, 2024
For this debugger related issue: dart-lang/sdk#57040, it was triaged to
area-languagebut should have been triaged toarea-vm.devoncarew commentedon Nov 8, 2024
Issue dart-lang/sdk#57055 was triaged into
area-test; it should have been triaged intoarea-vm.devoncarew commentedon Nov 11, 2024
From dart-lang/sdk#57064, it's difficult to know the right area to assign macros related issues.
lrhn commentedon Nov 12, 2024
I can generally say that
area-languageis more wrong than right. (There isn't much use forarea-languagein the SDK repo after the language repo was created, so it's unlikely thatarea-languagewill be correct.)