Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ just test-integration

# Run a specific integration test case (e.g., "grafted" test case)
TEST_CASE=grafted just test-integration

# Override ports if using different service ports (e.g., for local development)
POSTGRES_TEST_PORT=5432 ETHEREUM_TEST_PORT=8545 IPFS_TEST_PORT=5001 just test-integration
```

**⚠️ Test Verification Requirements:**
Expand All @@ -103,6 +106,8 @@ TEST_CASE=grafted just test-integration
- Integration tests take significant time (several minutes)
- Tests automatically reset the database between runs
- Logs are written to `tests/integration-tests/graph-node.log`
- **If a test hangs for >10 minutes**, it's likely stuck - kill with `pkill -9 integration_tests` and check logs
- CI uses the default ports (3011, 3021, 3001) - local development can override with environment variables

### Code Quality
```bash
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,44 @@ Very large `graph-node` instances can also be configured using a
the `graph-node` needs to connect to multiple chains or if the work of
indexing and querying needs to be split across [multiple databases](./docs/config.md).

#### Log Storage

`graph-node` supports storing and querying subgraph logs through multiple backends:

- **File**: Local JSON Lines files (recommended for local development)
- **Elasticsearch**: Enterprise-grade search and analytics (for production)
- **Loki**: Grafana's log aggregation system (for production)
- **Disabled**: No log storage (default)

**Quick example (file-based logs for local development):**
```bash
mkdir -p ./graph-logs

cargo run -p graph-node --release -- \
--postgres-url $POSTGRES_URL \
--ethereum-rpc mainnet:archive:https://... \
--ipfs 127.0.0.1:5001 \
--log-store-backend file \
--log-store-file-dir ./graph-logs
```

Logs are queried via GraphQL at `http://localhost:8000/graphql`:
```graphql
query {
_logs(subgraphId: "QmYourSubgraphHash", level: ERROR, first: 10) {
timestamp
level
text
}
}
```

**For complete documentation**, see the **[Log Store Guide](./docs/log-store.md)**, which covers:
- How to configure each backend (Elasticsearch, Loki, File)
- Complete GraphQL query examples
- Choosing the right backend for your use case
- Performance considerations and best practices

## Contributing

Please check [CONTRIBUTING.md](CONTRIBUTING.md) for development flow and conventions we use.
Expand Down
52 changes: 52 additions & 0 deletions docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,3 +302,55 @@ those.
Disabling the store call cache may significantly impact performance; the actual impact depends on
the average execution time of an `eth_call` compared to the cost of a database lookup for a cached result.
(default: false)

## Log Store Configuration

`graph-node` supports storing and querying subgraph logs through multiple backends: Elasticsearch, Loki, local files, or disabled.

**For complete log store documentation**, including detailed configuration, querying examples, and choosing the right backend, see the **[Log Store Guide](log-store.md)**.

### Quick Reference

**Backend selection:**
- `GRAPH_LOG_STORE_BACKEND`: `disabled` (default), `elasticsearch`, `loki`, or `file`

**Elasticsearch:**
- `GRAPH_LOG_STORE_ELASTICSEARCH_URL`: Elasticsearch endpoint URL (required)
- `GRAPH_LOG_STORE_ELASTICSEARCH_USER`: Username (optional)
- `GRAPH_LOG_STORE_ELASTICSEARCH_PASSWORD`: Password (optional)
- `GRAPH_LOG_STORE_ELASTICSEARCH_INDEX`: Index name (default: `subgraph`)

**Loki:**
- `GRAPH_LOG_STORE_LOKI_URL`: Loki endpoint URL (required)
- `GRAPH_LOG_STORE_LOKI_TENANT_ID`: Tenant ID (optional)

**File-based:**
- `GRAPH_LOG_STORE_FILE_DIR`: Log directory (required)
- `GRAPH_LOG_STORE_FILE_MAX_SIZE`: Max file size in bytes (default: 104857600 = 100MB)
- `GRAPH_LOG_STORE_FILE_RETENTION_DAYS`: Retention period (default: 30)

**Deprecated variables** (will be removed in future versions):
- `GRAPH_ELASTICSEARCH_URL` → use `GRAPH_LOG_STORE_ELASTICSEARCH_URL`
- `GRAPH_ELASTICSEARCH_USER` → use `GRAPH_LOG_STORE_ELASTICSEARCH_USER`
- `GRAPH_ELASTICSEARCH_PASSWORD` → use `GRAPH_LOG_STORE_ELASTICSEARCH_PASSWORD`
- `GRAPH_ELASTIC_SEARCH_INDEX` → use `GRAPH_LOG_STORE_ELASTICSEARCH_INDEX`

### Example: File-based Logs for Local Development

```bash
mkdir -p ./graph-logs
export GRAPH_LOG_STORE_BACKEND=file
export GRAPH_LOG_STORE_FILE_DIR=./graph-logs

graph-node \
--postgres-url postgresql://graph:pass@localhost/graph-node \
--ethereum-rpc mainnet:https://... \
--ipfs 127.0.0.1:5001
```

See the **[Log Store Guide](log-store.md)** for:
- Detailed configuration for all backends
- How log stores work internally
- GraphQL query examples
- Choosing the right backend for your use case
- Best practices and troubleshooting
Loading