Skip to content

feat(realtime): Allow sending text, image and audio conversation items"#8524

Merged
mudler merged 2 commits intomudler:masterfrom
richiejp:feat/realtime-streaming
Feb 12, 2026
Merged

feat(realtime): Allow sending text, image and audio conversation items"#8524
mudler merged 2 commits intomudler:masterfrom
richiejp:feat/realtime-streaming

Conversation

@richiejp
Copy link
Collaborator

@richiejp richiejp commented Feb 12, 2026

  • fix(realtime): Use locked websocket for concurrent access
  • fix(realtime): Use sample rate set in session
  • fix(config): Allow pipelines to have no model parameters

The above are already submitted in a separate PR which blocks this

  • feat(realtime): Allow sending text, image and audio conversation items

Description

Allow the client to add multi-modal conversation items. For example this can be used to submit pictures of the client's desktop, photos from their phone camera or a robot's.

Being able to submit text could also be very useful for tool calling assistants.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

@netlify
Copy link

netlify bot commented Feb 12, 2026

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 8dca94b
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/698e178b9235760008248b97
😎 Deploy Preview https://deploy-preview-8524--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

mudler
mudler previously approved these changes Feb 12, 2026
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp richiejp force-pushed the feat/realtime-streaming branch from 18a3457 to 4a91d4b Compare February 12, 2026 17:20
@richiejp richiejp marked this pull request as ready for review February 12, 2026 17:20
@mudler mudler enabled auto-merge (squash) February 12, 2026 18:10
@mudler mudler merged commit f6c80a6 into mudler:master Feb 12, 2026
36 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants