Skip to content

Fix raw markdown shown in image carousel caption#320754

Merged
mjbvz merged 4 commits into
microsoft:mainfrom
arun-357:fix/image-carousel-caption-markdown
Jun 10, 2026
Merged

Fix raw markdown shown in image carousel caption#320754
mjbvz merged 4 commits into
microsoft:mainfrom
arun-357:fix/image-carousel-caption-markdown

Conversation

@arun-357

Copy link
Copy Markdown
Contributor

Fixes #307446

Problem

The image carousel's bottom caption shows raw markdown instead of readable text, e.g.:

Viewed image [](file:///Users/pierceboggan/Documents/GitHub/coloring-book/homepage.png)

Root cause:

  • Tool invocations carry invocationMessage/pastTenseMessage typed string | IMarkdownString. Copilot's view-image tool produces markdown like Viewed image [](file:///…/homepage.png) (the empty-text link renders as a file widget in chat, but is raw syntax anywhere else).
  • chatImageExtraction.ts flattened the message to its raw markdown source via .value and stored that as the image caption.
  • The carousel editor renders captions as plain text (captionText.textContent = …, per its caption?: string contract), so the markdown syntax was displayed verbatim.

Fix

Preserve the message as string | IMarkdownString in chatImageExtraction.ts (common layer), and convert it to display text at the presentation layer in chatImageCarouselService.ts with a small toCaptionText() helper:

stripIcons(renderAsPlaintext(caption, { useLinkFormatter: true }))

This is the same conversion the chat UI already applies when rendering these tool messages as labels (see chatToolProgressPart.ts), so the carousel caption now matches what the user sees in chat: Viewed image homepage.png (useLinkFormatter resolves empty-text links to the file basename).

Notes:

  • The conversion lives in the browser layer because renderAsPlaintext is in vs/base/browser and chatImageExtraction.ts is common-layer code.
  • The carousel editor (imageCarouselEditor.ts) is intentionally untouched — its plain-text caption contract is correct; the chat layer was handing it markdown.
  • Plain-string messages pass through verbatim; images without a tool message still show no caption.

Tests

  • Added a regression test in chatImageCarouselService.test.ts (strips markdown from tool invocation message captions) using the exact message shape from the issue, plus a passthrough assertion for plain-string messages.
  • Updated chatImageExtraction.test.ts assertions for the caption type change (captions from IMarkdownString messages now assert on .value).

Before / After

Before:
markdown1

After:
markdown2

Copilot AI review requested due to automatic review settings June 10, 2026 11:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates chat image caption handling so tool invocation captions can carry Markdown metadata through extraction, while the carousel UI renders captions as plain text (matching chat rendering behavior).

Changes:

  • Widen extracted image caption type to support IMarkdownString and propagate Markdown captions from tool invocations.
  • Convert captions to plain text in the image carousel (renderAsPlaintext + stripIcons) to remove Markdown/link syntax.
  • Add/adjust tests for Markdown caption handling and text stripping.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
src/vs/workbench/contrib/chat/test/common/chatImageExtraction.test.ts Updates assertions to account for extracted captions being IMarkdownString.
src/vs/workbench/contrib/chat/test/browser/chatImageCarouselService.test.ts Adds a test ensuring Markdown is stripped from tool invocation captions in carousel output.
src/vs/workbench/contrib/chat/common/chatImageExtraction.ts Propagates Markdown captions and widens the extracted image caption type.
src/vs/workbench/contrib/chat/browser/chatImageCarouselService.ts Normalizes captions to plain text for display in carousel sections.

Comment thread src/vs/workbench/contrib/chat/test/common/chatImageExtraction.test.ts Outdated
Comment thread src/vs/workbench/contrib/chat/test/common/chatImageExtraction.test.ts Outdated
Comment thread src/vs/workbench/contrib/chat/test/common/chatImageExtraction.test.ts Outdated
Comment thread src/vs/workbench/contrib/chat/common/chatImageExtraction.ts
@arun-357

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree

@stasyu2009-ux

Copy link
Copy Markdown

В коді виникає проблема треба убрати

@arun-357

arun-357 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

В коді виникає проблема треба убрати

hey @stasyu2009-ux, can you point me to the problem.

@stasyu2009-ux

Copy link
Copy Markdown

Проблема в самому коді знаходиться

@stasyu2009-ux

Copy link
Copy Markdown

Запусти код і побачиш проблему

@arun-357

arun-357 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Запусти код і побачиш проблему

The code runs successfully on my end, and the fix is working as expected. Could you share the error you're encountering on your side ? @stasyu2009-ux

I dont see any new error / warning diffrent from main and my dev branch as well.

markdown.mov

@mjbvz mjbvz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mjbvz mjbvz enabled auto-merge June 10, 2026 16:46
@mjbvz mjbvz merged commit 677bdd2 into microsoft:main Jun 10, 2026
25 checks passed
@vs-code-engineering vs-code-engineering Bot added this to the 1.125.0 milestone Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Carousel view has invalid markdown rendering for "Viewed image..."

5 participants