Skip to content

fix: close stream after upstream ends to prevent client hang #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
octo-patch wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: close stream after upstream ends to prevent client hang #186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 18 additions & 10 deletions src/components/ConversationView/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,18 +259,26 @@ const ConversationView = () => {
const reader = data.getReader();
const decoder = new TextDecoder("utf-8");
let done = false;
while (!done) {
const { value, done: readerDone } = await reader.read();
if (value) {
const char = decoder.decode(value);
if (char) {
assistantMessage.content = assistantMessage.content + char;
messageStore.updateMessage(assistantMessage.id, {
content: assistantMessage.content,
});
try {
while (!done) {
const { value, done: readerDone } = await reader.read();
if (value) {
const char = decoder.decode(value);
if (char) {
assistantMessage.content = assistantMessage.content + char;
Comment on lines +264 to +268

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TextDecoder.decode(value) is called without { stream: true } inside a loop reading arbitrary byte chunks. This can corrupt multibyte UTF-8 characters that span chunk boundaries. Use streaming decode (decoder.decode(value, { stream: true })) and flush after the loop to ensure correct character reconstruction.

Copilot uses AI. Check for mistakes.
messageStore.updateMessage(assistantMessage.id, {
content: assistantMessage.content,
});
}
}
done = readerDone;
}
done = readerDone;
} catch (error) {
messageStore.updateMessage(assistantMessage.id, {
content: assistantMessage.content || "Failed to receive response. Please check your API endpoint configuration.",
status: "FAILED",
});
return;
Comment on lines +262 to +281

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reader lock is never released/canceled. In the error path you return inside catch, and in the success path you exit the loop without calling reader.releaseLock() (or reader.cancel() when failing). Wrapping the read loop in try/catch/finally and releasing/canceling the reader in finally will avoid leaving the stream locked and can help ensure the underlying connection is torn down promptly on errors.

Copilot uses AI. Check for mistakes.
}
messageStore.updateMessage(assistantMessage.id, {
status: "DONE",
Expand Down
14 changes: 12 additions & 2 deletions src/pages/api/chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,10 @@ const handler = async (req: NextRequest) => {
try {
const json = JSON.parse(data);
const text = json.choices[0].delta?.content;
const queue = encoder.encode(text);
controller.enqueue(queue);
if (text) {
const queue = encoder.encode(text);
controller.enqueue(queue);
}
} catch (e) {
controller.error(e);
}
Expand All @@ -151,6 +153,14 @@ const handler = async (req: NextRequest) => {
for await (const chunk of remoteRes.body as any) {
parser.feed(decoder.decode(chunk));
}
// Ensure the stream is closed after all upstream chunks are consumed.
// Some providers (e.g. Ollama native API) may not send a [DONE] sentinel,
// which would leave the ReadableStream open and the client hanging indefinitely.
try {
controller.close();
Comment on lines +156 to +160

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TextDecoder.decode(chunk) is used without { stream: true } while iterating arbitrary Uint8Array chunks. If a multibyte UTF-8 code point is split across chunks, decoding can introduce replacement characters and potentially break SSE parsing/JSON parsing. Consider decoding in streaming mode (decoder.decode(chunk, { stream: true })) and flushing once at the end (before closing) to handle boundary-split characters correctly.

Copilot uses AI. Check for mistakes.
} catch {
// Already closed by the [DONE] handler — ignore.
}
},
});

Expand Down
Loading