Set up the rest of the starting docs#3
Merged
Merged
Conversation
…ns, and use cases documents
…ork documentation
…s approach and open questions
…ion in approaches documentation
olaservo
added a commit
that referenced
this pull request
Feb 27, 2026
Add new references, findings, and discussion outcomes from the Feb 26 office hours (Discussion #2316): - Add MCPB and agentskills#110 to related work - Document push vs. pull update model comparison (FastMCP vs. PR #16) - Add skills-as-resources direction under Approach 3 (client helper tools, three-part framing, hinting workaround, next steps) - Add dependency declaration reference to Q4 and Use Case #3 - Add new question on skills/resources discovery via Server Cards Co-Authored-By: Claude Opus 4.6 <[email protected]>
This was referenced Mar 3, 2026
olaservo
added a commit
to olaservo/experimental-ext-skills
that referenced
this pull request
Apr 27, 2026
…ol#2/modelcontextprotocol#3 write-up Adds Scenario modelcontextprotocol#3 — `hf-train-with-monitoring` — testing whether the host's <available_skills> catalog drives multi-skill activation when one skill mentions another by topic without an explicit skill:// URI reference. Also documents Scenario modelcontextprotocol#2 (hf-jobs dry-run) and modelcontextprotocol#3 in docs/experimental-findings.md. Plan evaluator changes - expected_skill_uris (list) supersedes expected_skill_uri (singular, still accepted via backward compat) - skill-read-before-plan passes only when EVERY listed skill's SKILL.md was read before the gate; banner note carries read=[...] missed=[...] for diagnostics - raw payload gains skill_md_indices + missed_skills Scenario validator - Either expected_skill_uri or expected_skill_uris must be present (was: expected_skill_uri required as a base field) New tests (45 total) - Multi-skill happy path, partial activation (one skill missed), ordering (one skill after gate), backward-compat for legacy single-URI YAML Findings (docs/experimental-findings.md) - Scenario modelcontextprotocol#2 — four-client matrix on hf-mcp-server, all PASS; documents the prompt/skill conflict that surfaced in early runs (later fixed by reframing) and the goose gate_tools fix - Scenario modelcontextprotocol#3 — cross-skill composition works for fast-agent / codex / goose; gemini-cli misses cross-skill activation across both flash and pro tiers, even when the model would otherwise cover the prescriptions via training-data synthesis. Suggests host-side ranking of <available_skills> rather than model capability is the constraint Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
olaservo
added a commit
to olaservo/experimental-ext-skills
that referenced
this pull request
Apr 27, 2026
…rotocol#3/modelcontextprotocol#4 as N=1 samples Adds a one-line note under each scenario header so readers don't mistake single-trial outcomes for stable behavioral claims. Re-running flipped haiku from FAIL to PASS on the dispose-rule criterion in Scenario modelcontextprotocol#4, which is exactly the noise the caveat warns about. Pass-rate replication (N=3+ per cell) is the proper fix and pending in a follow-up — N-trial wrapper in each client harness, then re-run the matrix and rewrite the findings to report rates rather than single-sample binaries. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
olaservo
added a commit
to olaservo/experimental-ext-skills
that referenced
this pull request
Apr 27, 2026
Drop the "Scenarios modelcontextprotocol#2/modelcontextprotocol#3/modelcontextprotocol#4 are N=1, replication pending" snapshot — that fact rots as findings get re-run. Keep just the principle: treat individual outcomes as samples, not stable behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
olaservo
added a commit
to olaservo/experimental-ext-skills
that referenced
this pull request
Apr 28, 2026
…extprotocol#3/modelcontextprotocol#4 Three new YAMLs that exercise the plan-kind evaluator path: - hf-jobs-plan.yaml (modelcontextprotocol#2): probes structured plan output (PEP 723 UV script + HF_TOKEN secret forwarding + Trackio instrumentation) on hf-mcp-server with HF_JOBS_DRY_RUN=true, so submissions are intercepted server-side and graded from the call args - hf-train-with-monitoring.yaml (modelcontextprotocol#3): cross-skill composition -- agent must independently activate both huggingface-llm-trainer and huggingface-trackio from the catalog, with required phrases drawn from both skills' references - transformers-js-demo.yaml (modelcontextprotocol#4): code-output domain (browser HTML/JS artifact). No tool execution; phrase-grep checks @huggingface/transformers package + pipeline API + dispose() rule All three use gate_tools to anchor skill-read-before-plan against meaningful actions instead of housekeeping calls (todo writes, catalog enumeration), so agentic planners aren't penalized.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transitioning content from google doc to this repo.