You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[BUG] OpenAI-compatible provider + Gemini: report issues have null 'finding' and 'category' (explanation/severity OK) #304
When the active provider is openai pointed at Google's OpenAI-compatible endpoint (https://generativelanguage.googleapis.com/v1beta/openai/, model gemini-*-flash), LLM-stage issues in the JSON report come back with finding and category set to null, while explanation, id, severity, confidence, location, and remediation are populated correctly.
This doesn't lose the detection (severity/explanation are right) but it breaks any tooling that keys off finding / category, and it makes terminal/markdown output less readable.
Example
Scanning a test skill that exfiltrates ~/.config/gh/hosts.yml:
severity=CRITICAL with a correct explanation, but finding and category are null.
Likely cause
The structured-output schema mapping for OpenAI-compatible providers seems to leave these two fields unpopulated when the backend is Gemini (as opposed to stock OpenAI). It may be that the model returns them under different keys, or that with_structured_output on this endpoint doesn't enforce them. I haven't traced the exact spot.
Impact
Low severity — detection is intact — but it's a real interop rough edge for anyone using SkillSpector's growing OpenAI-compatible provider support with non-OpenAI backends (Gemini, and likely others). Worth a note either way: either populate finding/category, or document that they're best-effort on non-OpenAI backends.
@jayanth922 sorry for the long silence — go ahead if you still want it.
One caveat before you spend time on it: I cannot re-verify the current behaviour, as I no longer have a Gemini endpoint configured here. The report was filed in July, so please reproduce it against main first and close it if it no longer holds.
Implementation is now in progress in PR #616, which fills the shared LLM-finding conversion's category, pattern, and finding fields and adds regression coverage. The PR is open and has not merged, so this issue remains open pending review, merge, and verification of the reported output.
Resolution verified on 2026-10-04: PR #616 has merged, and GitHub automatically closed this issue.
The shared LLM-to-report conversion now populates category, pattern, and finding metadata, including the SSD-3 case in this report. The merged change also supplies TP4 metadata and tests sanitization of copied model text.
Verification on merged main (701baea): the LLM conversion, report sanitizer, and MCP tool-poisoning suites passed (393 passed, 6 deselected). These are local regression tests, not a live Gemini endpoint test.
Activity
jayanth922 commentedon Jul 23, 2026
@Mark2Mac I would like to contribute to this issue
Mark2Mac commentedon Aug 31, 2026
@jayanth922 sorry for the long silence — go ahead if you still want it.
One caveat before you spend time on it: I cannot re-verify the current behaviour, as I no longer have a Gemini endpoint configured here. The report was filed in July, so please reproduce it against
mainfirst and close it if it no longer holds.rng1995 commentedon Sep 23, 2026
Implementation is now in progress in PR #616, which fills the shared LLM-finding conversion's category, pattern, and finding fields and adds regression coverage. The PR is open and has not merged, so this issue remains open pending review, merge, and verification of the reported output.
rng1995 commentedon Oct 4, 2026
Resolution verified on 2026-10-04: PR #616 has merged, and GitHub automatically closed this issue.
The shared LLM-to-report conversion now populates category, pattern, and finding metadata, including the SSD-3 case in this report. The merged change also supplies TP4 metadata and tests sanitization of copied model text.
Verification on merged main (701baea): the LLM conversion, report sanitizer, and MCP tool-poisoning suites passed (393 passed, 6 deselected). These are local regression tests, not a live Gemini endpoint test.