-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Category: Performance / Observability
Component: HelpRequest, Record, thread-local drop-off system
🧠 Purpose
Evaluate the memory footprint and GC behavior of HelpRequest and Record under long-running workloads, particularly in threaded environments where many tasks are spawned, completed, and dropped into the data center.
✅ Objectives
- Confirm that
_work_callableis purged immediately after use. - Ensure no lingering object graphs remain after
HelpRequestcompletion. - Validate whether
dispose()meaningfully reduces memory pressure when used. - Inspect GC behavior with and without calling
dispose()onHelpRequestandRecord. - Test
ConcurrentSetpromotion inRecord.factory_idfor multiple-agent access. - Validate that thread-local journaling does not retain unnecessary references.
🔬 Test Scenarios
1. High-Frequency Task Creation
- Create and complete 100k+
HelpRequestinstances across multiple agent threads. - Drop all records into the thread’s
data_center. - Monitor memory before and after via
tracemallocorpsutil.
2. Mixed Terminal States
- Simulate failed, cancelled, and completed requests.
- Confirm that all states lead to callable cleanup and eventual GC collection.
3. Dispose vs No Dispose
- Compare memory usage with and without explicit
dispose()calls onHelpRequestandRecord. - Identify if
RecordorConcurrentSetretain memory without disposal.
4. Monkeypatch Impact
- Test monkeypatched thread attributes (
_help_request,_data_center, etc). - Ensure no memory leak from lingering thread-local attributes post-task.
🛠️ Tools
gc.get_objects()/gc.collect()tracemallocsnapshots- Thread dump / post-mortem heap analysis (e.g.,
objgraph,memory_profiler) - Optional: stress tests with
pytest-benchmark
📎 Artifacts to Capture
- Peak memory usage per agent thread
- Growth pattern over time (plateau or leak)
- Object count of
HelpRequest,Record, andConcurrentSet - GC unreachable count before and after test batch
- Sample task that was most memory-intensive (if possible)
🔄 Acceptance Criteria
- No memory growth when tasks are completed and dropped.
_work_callableis purged consistently even withoutdispose().- Records can be safely retained without disposal unless holding complex sets.
- Thread-local patching does not retain long-lived garbage.
💬 Notes
This test will verify that agentic task routing and record storage scale efficiently in memory-constrained environments, and help ensure Melder and ThreadFactory remain lightweight even under sustained load.
Metadata
Metadata
Assignees
Labels
No labels