Skip to content

[Bug]: Guided Learning choice options are lost and correct answers may be graded incorrectly #603

Description

@spring618

Do you need to file an issue?

  • I have searched the existing issues and this bug is not already filed.
  • I believe this is a legitimate bug, not just a question or feature request.

Describe the bug

Guided Learning choice questions have an inconsistent data contract between
mastery_quiz, ask_user, deterministic grading, and the Question Bank.

This causes two user-visible failures:

  1. Some Question Bank entries display only A, B, C, and D, without the
    corresponding option text.
  2. A correct card selection can be graded as wrong when mastery_quiz stores
    an option-body fragment such as Step 6 as expected_answer, while
    ask_user returns the option label C.

In an affected local database, notebook_entries.options_json contained:

{"A":"A","B":"B","C":"C","D":"D"}

The matching turn_events row still contained the complete descriptions in
the ask_user arguments. In two other affected entries, the stored expected
answers were Step 4 and Step 6, while the user's correct card response was
C; both were recorded as incorrect.

Steps to reproduce

  1. Run DeepTutor with the Docker image and configure a model.
  2. Create or open a Guided Learning / Mastery Path containing a memory or
    procedure knowledge point.
  3. Continue until the tutor generates a multiple-choice question through
    mastery_quiz and presents it through ask_user.
  4. Answer the interactive choice card.
  5. Open Learning Space → Question Bank.
  6. For turns where the model passed options: ["A", "B", "C", "D"] to
    mastery_quiz, observe that the Question Bank has lost the option bodies.
  7. For turns where expected_answer is an option-body fragment instead of a
    label, select the correct label and observe that it can be graded wrong.

Expected behavior

  • mastery_quiz.options should persist every complete option body.
  • Bare label-only option arrays should be rejected and retried.
  • Choice answers should be normalized to their stable labels before
    deterministic grading and Question Bank persistence.
  • The Question Bank should always render the same option text shown on the
    original ask_user card.

Related module

Guided Learning

Configuration used

  • Docker image: ghcr.io/hkuds/deeptutor:latest
  • Guided Learning with an attached knowledge base
  • Interactive questions presented through ask_user

Additional information

Likely affected code:

  • deeptutor/capabilities/mastery/tools.py
  • deeptutor/capabilities/mastery/prompts/en/system.md
  • deeptutor/capabilities/mastery/prompts/zh/system.md

A fix and regression tests are ready and will be submitted in a linked PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions