> ## Documentation Index
> Fetch the complete documentation index at: https://docs.depthsai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# S3 Configuration

> Configure S3 with environment variables, start the server, and query sealed days from S3 using the CLI.

# S3 Configuration

Depths v0.1.1 can seal each UTC day of data and ship it to S3 or any S3-compatible store. S3-backup is optional : local-only works fine. When S3 is set by environment variables, the server ships sealed days and verifies row counts. You can then read sealed days from S3. The querying experience
via the query endpoints stays identical, allowing seamless analysis over local and s3-backed telemetry.

## When to enable S3

* Durable storage for historical days
* Read sealed days from object storage instead of the ingest box
* S3-compatible endpoints (MinIO, DigitalOcean Spaces, etc.)

## Environment variables

Set these before you run `depths init` and `depths start`.

### Required

| Variable                                                             | Meaning                                                              |
| -------------------------------------------------------------------- | -------------------------------------------------------------------- |
| `S3_BUCKET`                                                          | Bucket name                                                          |
| `AWS_ACCESS_KEY_ID` or `S3_ACCESS_KEY_ID`                            | Access key                                                           |
| `AWS_SECRET_ACCESS_KEY` or `S3_SECRET_KEY` or `S3_SECRET_ACCESS_KEY` | Secret                                                               |
| `AWS_REGION` or `S3_REGION`                                          | Region (leave empty for some S3-compatible vendors)                  |
| `AWS_ENDPOINT_URL` or `S3_URL`                                       | Endpoint URL, e.g. `https://s3.amazonaws.com` or `http://minio:9000` |

### Optional

| Variable            | Meaning                                |
| ------------------- | -------------------------------------- |
| `S3_PREFIX`         | Key prefix, e.g. `depths-prod`         |
| `AWS_SESSION_TOKEN` | Session token if using temporary creds |

> For S3-compatible endpoints using `http://`, reads use `AWS_ALLOW_HTTP=true` internally. Prefer TLS in production.

## Set variables

<CodeGroup>
  ```bash macOS/Linux theme={null}
  export S3_BUCKET="my-bucket"
  export AWS_ACCESS_KEY_ID="…"
  export AWS_SECRET_ACCESS_KEY="…"
  export AWS_REGION="ap-south-1"
  export AWS_ENDPOINT_URL="https://s3.amazonaws.com"
  # optional
  export S3_PREFIX="depths-prod"
  export AWS_SESSION_TOKEN=""
  ```

  ```powershell Windows theme={null}
  $env:S3_BUCKET="my-bucket"
  $env:AWS_ACCESS_KEY_ID="…"
  $env:AWS_SECRET_ACCESS_KEY="…"
  $env:AWS_REGION="ap-south-1"
  $env:AWS_ENDPOINT_URL="https://s3.amazonaws.com"
  # optional
  $env:S3_PREFIX="depths-prod"
  $env:AWS_SESSION_TOKEN=""
  ```
</CodeGroup>

## Start with shipping enabled

<CodeGroup>
  ```bash macOS/Linux theme={null}
  depths init
  depths start
  ```

  ```powershell Windows theme={null}
  depths init
  depths start
  ```
</CodeGroup>

What happens:

1. During the day, rows are appended to local Delta tables.
2. On UTC day rollover, the server seals the day, uploads the tables to S3, verifies row counts, then cleans local copies when verification passes.

## Bucket layout

Days are stored under:

```
s3://<bucket>/<prefix>/<instance_id>/days/<YYYY-MM-DD>/
```

Each day contains six Delta tables under `otel/`:
`spans`, `span_events`, `span_links`, `logs`, `metrics_points`, `metrics_hist`.

## Verify shipping

Health:

```bash theme={null}
curl -s http://localhost:4318/healthz | jq
```

Index (recent lines):

<CodeGroup>
  ```bash macOS/Linux theme={null}
  tail -n 20 ./depths_data/default/index/days.jsonl
  ```

  ```powershell Windows theme={null}
  Get-Content .\depths_data\default\index\days.jsonl -Tail 20
  ```
</CodeGroup>

You should see the phases per day: `sealed` → `uploaded` → `verified` → `cleaned` with row counts.

## Read from S3

Use the CLI to read directly from S3, or let it auto-select.

<CodeGroup>
  ```bash macOS/Linux theme={null}
  depths view -t spans -S s3
  depths view -t spans -S auto
  ```

  ```powershell Windows theme={null}
  depths view -t spans -S s3
  depths view -t spans -S auto
  ```
</CodeGroup>

The HTTP query endpoints also accept a `storage` parameter (`auto`, `local`, `s3`) if you prefer API access.

## Common pitfalls

* **Partial env** : any missing required variable disables shipping; set env before `depths start`.
* **Endpoint** : `http://` works for compatible stores; prefer `https://` for production.
* **Permissions** : the key needs `PutObject`, `List`, and `Get` in the bucket and prefix.