Skip to content

Commit 690a2a9

Browse files
Update AudioFlinger docs
1 parent ea35e0e commit 690a2a9

File tree

1 file changed

+257
-13
lines changed

1 file changed

+257
-13
lines changed

docs/frameworks/audioflinger.md

Lines changed: 257 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,27 @@
11
# AudioFlinger
22

3-
MicroPythonOS provides a centralized audio service called **AudioFlinger**, inspired by Android's architecture. It manages audio playback across different hardware outputs with priority-based audio focus control.
3+
MicroPythonOS provides a centralized audio service called **AudioFlinger**, inspired by Android's architecture. It manages audio playback and recording across different hardware outputs with priority-based audio focus control.
44

55
## Overview
66

77
AudioFlinger provides:
88

99
- **Priority-based audio focus** - Higher priority streams interrupt lower priority ones
1010
- **Multiple audio devices** - I2S digital audio, PWM buzzer, or both
11-
- **Background playback** - Runs in separate thread
11+
- **Background playback/recording** - Runs in separate thread
1212
- **WAV file support** - 8/16/24/32-bit PCM, mono/stereo, auto-upsampling
13+
- **WAV recording** - 16-bit mono PCM from I2S microphone
1314
- **RTTTL ringtone support** - Full Ring Tone Text Transfer Language parser
1415
- **Thread-safe** - Safe for concurrent access
1516
- **Hardware-agnostic** - Apps work across all platforms without changes
1617

1718
## Supported Audio Devices
1819

19-
- **I2S**: Digital audio output for WAV file playback (Fri3d badge, Waveshare board)
20+
- **I2S Output**: Digital audio output for WAV file playback (Fri3d badge, Waveshare board)
21+
- **I2S Input**: Microphone recording (Fri3d badge only)
2022
- **Buzzer**: PWM-based tone/ringtone playback (Fri3d badge only)
2123
- **Both**: Simultaneous I2S and buzzer support
22-
- **Null**: No audio (desktop/Linux)
24+
- **Null**: No audio (desktop/Linux - simulated for testing)
2325

2426
## Quick Start
2527

@@ -88,6 +90,130 @@ print(f"Current volume: {volume}")
8890
AudioFlinger.stop()
8991
```
9092

93+
## Recording Audio
94+
95+
AudioFlinger supports recording audio from an I2S microphone to WAV files.
96+
97+
### Basic Recording
98+
99+
```python
100+
import mpos.audio.audioflinger as AudioFlinger
101+
102+
# Check if microphone is available
103+
if AudioFlinger.has_microphone():
104+
# Record for 10 seconds
105+
success = AudioFlinger.record_wav(
106+
file_path="data/my_recording.wav",
107+
duration_ms=10000,
108+
sample_rate=16000,
109+
on_complete=lambda msg: print(msg)
110+
)
111+
112+
if success:
113+
print("Recording started...")
114+
else:
115+
print("Recording failed to start")
116+
```
117+
118+
### Recording Parameters
119+
120+
- **file_path**: Path to save the WAV file
121+
- **duration_ms**: Maximum recording duration in milliseconds (default: 60000 = 60 seconds)
122+
- **sample_rate**: Sample rate in Hz (default: 16000 - good for voice)
123+
- **on_complete**: Callback function called when recording finishes
124+
125+
### Stopping Recording
126+
127+
```python
128+
# Stop recording early
129+
AudioFlinger.stop()
130+
```
131+
132+
### Recording Constraints
133+
134+
1. **Cannot record while playing**: I2S can only be TX (output) or RX (input) at one time
135+
2. **Cannot start new recording while recording**: Only one recording at a time
136+
3. **Microphone required**: `has_microphone()` must return `True`
137+
138+
```python
139+
# Check recording state
140+
if AudioFlinger.is_recording():
141+
print("Currently recording...")
142+
143+
# Check playback state
144+
if AudioFlinger.is_playing():
145+
print("Currently playing...")
146+
```
147+
148+
### Complete Recording Example
149+
150+
```python
151+
from mpos.apps import Activity
152+
import mpos.audio.audioflinger as AudioFlinger
153+
import lvgl as lv
154+
import time
155+
156+
class SoundRecorderActivity(Activity):
157+
def onCreate(self):
158+
self.screen = lv.obj()
159+
self.is_recording = False
160+
161+
# Status label
162+
self.status = lv.label(self.screen)
163+
self.status.set_text("Ready")
164+
self.status.align(lv.ALIGN.TOP_MID, 0, 20)
165+
166+
# Record button
167+
self.record_btn = lv.button(self.screen)
168+
self.record_btn.set_size(120, 50)
169+
self.record_btn.align(lv.ALIGN.CENTER, 0, 0)
170+
self.record_label = lv.label(self.record_btn)
171+
self.record_label.set_text("Record")
172+
self.record_label.center()
173+
self.record_btn.add_event_cb(self.toggle_recording, lv.EVENT.CLICKED, None)
174+
175+
# Check microphone availability
176+
if not AudioFlinger.has_microphone():
177+
self.status.set_text("No microphone")
178+
self.record_btn.add_flag(lv.obj.FLAG.HIDDEN)
179+
180+
self.setContentView(self.screen)
181+
182+
def toggle_recording(self, event):
183+
if self.is_recording:
184+
AudioFlinger.stop()
185+
self.is_recording = False
186+
self.record_label.set_text("Record")
187+
self.status.set_text("Stopped")
188+
else:
189+
# Generate timestamped filename
190+
t = time.localtime()
191+
filename = f"data/recording_{t[0]}{t[1]:02d}{t[2]:02d}_{t[3]:02d}{t[4]:02d}.wav"
192+
193+
success = AudioFlinger.record_wav(
194+
file_path=filename,
195+
duration_ms=60000, # 60 seconds max
196+
sample_rate=16000,
197+
on_complete=self.on_recording_complete
198+
)
199+
200+
if success:
201+
self.is_recording = True
202+
self.record_label.set_text("Stop")
203+
self.status.set_text("Recording...")
204+
else:
205+
self.status.set_text("Failed to start")
206+
207+
def on_recording_complete(self, message):
208+
# Called from recording thread - update UI safely
209+
self.update_ui_threadsafe_if_foreground(self._update_ui_after_recording, message)
210+
211+
def _update_ui_after_recording(self, message):
212+
self.is_recording = False
213+
self.record_label.set_text("Record")
214+
self.status.set_text(message)
215+
```
216+
91217
## Audio Focus Priority
92218

93219
AudioFlinger implements a 3-tier priority-based audio focus system inspired by Android:
@@ -125,20 +251,29 @@ AudioFlinger.play_wav("alarm.wav", stream_type=AudioFlinger.STREAM_ALARM)
125251

126252
## Hardware Support Matrix
127253

128-
| Board | I2S | Buzzer | Notes |
129-
|-------|-----|--------|-------|
130-
| **Fri3d 2024 Badge** |GPIO 2, 47, 16 |GPIO 46 | Both devices available |
131-
| **Waveshare ESP32-S3** |GPIO 2, 47, 16 || I2S only |
132-
| **Linux/macOS** ||| No audio (desktop builds) |
254+
| Board | I2S Output | I2S Microphone | Buzzer | Notes |
255+
|-------|------------|----------------|--------|-------|
256+
| **Fri3d 2024 Badge** |||| Full audio support |
257+
| **Waveshare ESP32-S3** |||| I2S output only |
258+
| **Linux/macOS** || ✅ (simulated) | | Simulated recording for testing |
133259

134-
**I2S Pins:**
135-
- **BCLK** (Bit Clock): GPIO 2
260+
**I2S Output Pins (DAC/Speaker):**
261+
- **BCK** (Bit Clock): GPIO 2
136262
- **WS** (Word Select): GPIO 47
137263
- **DOUT** (Data Out): GPIO 16
138264

265+
**I2S Input Pins (Microphone):**
266+
- **SCLK** (Serial Clock): GPIO 17
267+
- **WS** (Word Select): GPIO 47 (shared with output)
268+
- **DIN** (Data In): GPIO 15
269+
139270
**Buzzer Pin:**
140271
- **PWM**: GPIO 46 (Fri3d badge only)
141272

273+
!!! note "I2S Limitation"
274+
The ESP32 I2S peripheral can only be in TX (output) or RX (input) mode at one time.
275+
You cannot play and record simultaneously.
276+
142277
## Configuration
143278

144279
Audio device preference is configured in the Settings app under **"Advanced Settings"**:
@@ -222,7 +357,7 @@ class SimpleMusicPlayerActivity(Activity):
222357

223358
## API Reference
224359

225-
### Functions
360+
### Playback Functions
226361

227362
**`play_wav(path, stream_type=STREAM_MUSIC, volume=None, on_complete=None)`**
228363

@@ -246,6 +381,39 @@ Play an RTTTL ringtone.
246381

247382
- **Returns:** `bool` - `True` if playback started, `False` if rejected
248383

384+
### Recording Functions
385+
386+
**`record_wav(file_path, duration_ms=None, on_complete=None, sample_rate=16000)`**
387+
388+
Record audio from I2S microphone to WAV file.
389+
390+
- **Parameters:**
391+
- `file_path` (str): Path to save WAV file (e.g., `"data/recording.wav"`)
392+
- `duration_ms` (int, optional): Recording duration in milliseconds. Default: 60000 (60 seconds)
393+
- `on_complete` (callable, optional): Callback function called when recording finishes
394+
- `sample_rate` (int, optional): Sample rate in Hz. Default: 16000 (good for voice)
395+
396+
- **Returns:** `bool` - `True` if recording started, `False` if rejected
397+
398+
- **Rejection reasons:**
399+
- No microphone available (`has_microphone()` returns `False`)
400+
- Currently playing audio (I2S can only be TX or RX)
401+
- Already recording
402+
403+
**`is_recording()`**
404+
405+
Check if audio is currently being recorded.
406+
407+
- **Returns:** `bool` - `True` if recording active, `False` otherwise
408+
409+
**`has_microphone()`**
410+
411+
Check if I2S microphone is available for recording.
412+
413+
- **Returns:** `bool` - `True` if microphone configured, `False` otherwise
414+
415+
### Volume and Control Functions
416+
249417
**`set_volume(volume)`**
250418

251419
Set playback volume.
@@ -261,7 +429,27 @@ Get current playback volume.
261429

262430
**`stop()`**
263431

264-
Stop currently playing audio.
432+
Stop currently playing audio or recording.
433+
434+
**`is_playing()`**
435+
436+
Check if audio is currently playing.
437+
438+
- **Returns:** `bool` - `True` if playback active, `False` otherwise
439+
440+
### Hardware Detection Functions
441+
442+
**`has_i2s()`**
443+
444+
Check if I2S audio output is available for WAV playback.
445+
446+
- **Returns:** `bool` - `True` if I2S configured, `False` otherwise
447+
448+
**`has_buzzer()`**
449+
450+
Check if buzzer is available for RTTTL playback.
451+
452+
- **Returns:** `bool` - `True` if buzzer configured, `False` otherwise
265453

266454
### Stream Type Constants
267455

@@ -387,6 +575,45 @@ ffmpeg -i input.wav -acodec pcm_s16le -ar 22050 -ac 1 output.wav
387575
- RTTTL requires hardware buzzer (Fri3d badge only)
388576
- I2S cannot produce RTTTL tones
389577

578+
### Recording Not Working
579+
580+
**Symptom**: `record_wav()` returns `False` or no audio recorded
581+
582+
**Possible causes:**
583+
584+
1. **No microphone available**
585+
```python
586+
if not AudioFlinger.has_microphone():
587+
print("No microphone on this device")
588+
```
589+
590+
2. **Currently playing audio**
591+
```python
592+
# I2S can only be TX or RX, not both
593+
if AudioFlinger.is_playing():
594+
AudioFlinger.stop()
595+
time.sleep_ms(100) # Wait for cleanup
596+
AudioFlinger.record_wav("recording.wav")
597+
```
598+
599+
3. **Already recording**
600+
```python
601+
if AudioFlinger.is_recording():
602+
print("Already recording")
603+
```
604+
605+
4. **Wrong board** - Waveshare doesn't have a microphone
606+
607+
### Recording Quality Issues
608+
609+
**Symptom**: Recording sounds distorted or has noise
610+
611+
**Solutions:**
612+
613+
1. **Use appropriate sample rate**: 16000 Hz is good for voice, 44100 Hz for music
614+
2. **Check microphone placement**: Keep microphone away from noise sources
615+
3. **Verify I2S pin configuration**: Check board file for correct pin assignments
616+
390617
## Performance Tips
391618

392619
### Optimizing WAV Files
@@ -425,6 +652,23 @@ AudioFlinger.play_wav(
425652
- Buffers are allocated during playback and freed after
426653
- Multiple simultaneous streams not supported (priority system prevents this)
427654

655+
## Desktop Testing
656+
657+
On desktop builds (Linux/macOS), AudioFlinger provides simulated recording for testing:
658+
659+
- **Microphone simulation**: Generates a 440Hz sine wave instead of real audio
660+
- **WAV file generation**: Creates valid WAV files that can be played back
661+
- **Real-time simulation**: Recording runs at realistic speed
662+
663+
This allows testing the Sound Recorder app and other recording features without hardware.
664+
665+
```python
666+
# Desktop simulation is automatic when machine.I2S is not available
667+
# The generated WAV file contains a 440Hz tone
668+
AudioFlinger.record_wav("test.wav", duration_ms=5000)
669+
# Creates a valid 5-second WAV file with 440Hz sine wave
670+
```
671+
428672
## See Also
429673

430674
- [Creating Apps](../apps/creating-apps.md) - Learn how to create MicroPythonOS apps

0 commit comments

Comments
 (0)