Python Bug: AzureAIAgent: Filename (title) from Azure AI Foundry url_citation is not mapped to AnnotationContent

I have an AI Agent configured in Azure AI Foundry, connected to a vector database via Azure AI Search as its knowledge base. Agent is instructed to provide citations.

When agent is invoked directly using AIProjectClient (azure-ai-projects SDK), the response's annotations include a url_citation dictionary containing both a url (document ID) and a title (the source filename).
However, when the same Azure AI Foundry agent is invoked via AzureAIAgent, this filename information (and few others) is lost in the resulting AnnotationContent.

Data Examples:

Raw Annotation Data from Agent (via AIProjectClient):
A single annotation object within the list of annotations from the direct agent call shows:

{
    "type": "url_citation",
    "text": "【3:2†source】",
    "start_index": 1492,
    "end_index": 1504,
    "url_citation": {
        "url": "doc_2",
        "title": "Moulding-System-ENGLISH.pdf" 
    }
}

(Note the title field containing the filename)

Resulting AnnotationContent in Semantic Kernel (via AzureAIAgent):
The corresponding AnnotationContent object within ChatMessageContent.items after AzureAIAgent processes the response:

AnnotationContent(
    inner_content=None, 
    ai_model_id=None, # ai_model_id should also be pulled from Azure
    metadata={},  # Metadata is empty
    content_type='annotation', 
    file_id=None, 
    quote='【3:2†source】',
    start_index=1317, # Example index
    end_index=1329,   # Example index
    url='doc_2',
    title=None,       # CRITICAL: title is None
    citation_type=None
)

(Note title=None and metadata={})

Expected Behavior:
The AnnotationContent object in Semantic Kernel should reflect the filename. For the example above, we would expect:

AnnotationContent.title to be "Moulding-System-ENGLISH.pdf".
OR, the url_citation dictionary {'url': 'doc_2', 'title': 'Moulding-System-ENGLISH.pdf'} to be accessible via AnnotationContent.metadata.

Actual Behavior:
The filename "Moulding-System-ENGLISH.pdf" is not accessible through the AnnotationContent object when using AzureAIAgent. Only the url (e.g., "doc_2") is preserved.

Request:
Could the AzureAIAgent be updated to ensure that the title (filename and other missing parameters) from Azure AI Foundry's url_citation annotations is correctly mapped and made accessible within Semantic Kernel's AnnotationContent object?

Maybe, to add "title" into semantic-kernel -> agents -> azure_ai -> agent_content_generation.py,
go to _generate_annotation_content and update it by adding title similar to this:

@experimental
def generate_annotation_content(
    annotation: MessageTextFilePathAnnotation | MessageTextFileCitationAnnotation | MessageTextUrlCitationAnnotation,
) -> AnnotationContent:

"""Generate annotation content."""

    file_id = None
    url = None
    title = None # added
    if isinstance(annotation, MessageTextFilePathAnnotation) and annotation.file_path is not None:
        file_id = annotation.file_path.file_id
    elif isinstance(annotation, MessageTextFileCitationAnnotation) and annotation.file_citation is not None:
        file_id = annotation.file_citation.file_id
    elif isinstance(annotation, MessageTextUrlCitationAnnotation) and annotation.url_citation is not None:
        url = annotation.url_citation.url if annotation.url_citation.url else None
        title = annotation.url_citation.title if annotation.url_citation.title else None # added

    return AnnotationContent(
        file_id=file_id,
        quote=annotation.text,
        start_index=annotation.start_index if annotation.start_index is not None else None,
        end_index=annotation.end_index if annotation.end_index is not None else None,
        url=url,
        title= title, # added
    )

Then we will see in TextContent of the response items the titles like:

AnnotationContent(
    inner_content=None,
    ai_model_id=None,
    metadata={},
    content_type='annotation',
    file_id=None,
    quote=' ',
    start_index=305,
    end_index=317,
    url='doc_0',
    title='General Presentation.pptx',
    citation_type=None,
)

Semantic Kernel version: 1.30.0
Does it help?

MarekFabinski

Hi MarekFabinski, thanks for filing and for the details. We'll get this fixed ASAP.

Python: Preserve Citation Title in AnnotationContent from Azure AI Fo…

Navigation Menu

Repository navigation

Python Bug: AzureAIAgent: Filename (title) from Azure AI Foundry url_citation is not mapped to AnnotationContent #12133

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Python Bug: AzureAIAgent: Filename (title) from Azure AI Foundry url_citation is not mapped to AnnotationContent #12133

Description

Activity

seizeuniverse commented on May 16, 2025

moonbox3 commented on May 19, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions