Skip to content

📈 Issue: Memory Performance Testing for HelpRequest and Record #9

@Synaptic724

Description

@Synaptic724

Category: Performance / Observability
Component: HelpRequest, Record, thread-local drop-off system


🧠 Purpose

Evaluate the memory footprint and GC behavior of HelpRequest and Record under long-running workloads, particularly in threaded environments where many tasks are spawned, completed, and dropped into the data center.


✅ Objectives

  • Confirm that _work_callable is purged immediately after use.
  • Ensure no lingering object graphs remain after HelpRequest completion.
  • Validate whether dispose() meaningfully reduces memory pressure when used.
  • Inspect GC behavior with and without calling dispose() on HelpRequest and Record.
  • Test ConcurrentSet promotion in Record.factory_id for multiple-agent access.
  • Validate that thread-local journaling does not retain unnecessary references.

🔬 Test Scenarios

1. High-Frequency Task Creation

  • Create and complete 100k+ HelpRequest instances across multiple agent threads.
  • Drop all records into the thread’s data_center.
  • Monitor memory before and after via tracemalloc or psutil.

2. Mixed Terminal States

  • Simulate failed, cancelled, and completed requests.
  • Confirm that all states lead to callable cleanup and eventual GC collection.

3. Dispose vs No Dispose

  • Compare memory usage with and without explicit dispose() calls on HelpRequest and Record.
  • Identify if Record or ConcurrentSet retain memory without disposal.

4. Monkeypatch Impact

  • Test monkeypatched thread attributes (_help_request, _data_center, etc).
  • Ensure no memory leak from lingering thread-local attributes post-task.

🛠️ Tools

  • gc.get_objects() / gc.collect()
  • tracemalloc snapshots
  • Thread dump / post-mortem heap analysis (e.g., objgraph, memory_profiler)
  • Optional: stress tests with pytest-benchmark

📎 Artifacts to Capture

  • Peak memory usage per agent thread
  • Growth pattern over time (plateau or leak)
  • Object count of HelpRequest, Record, and ConcurrentSet
  • GC unreachable count before and after test batch
  • Sample task that was most memory-intensive (if possible)

🔄 Acceptance Criteria

  • No memory growth when tasks are completed and dropped.
  • _work_callable is purged consistently even without dispose().
  • Records can be safely retained without disposal unless holding complex sets.
  • Thread-local patching does not retain long-lived garbage.

💬 Notes

This test will verify that agentic task routing and record storage scale efficiently in memory-constrained environments, and help ensure Melder and ThreadFactory remain lightweight even under sustained load.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions