# OpenStrate — 3D Reconstruction & Perception

> **Public URL for agents (plain Markdown):** https://openstrate.com/openstrate-agent-skill.md  
> **Short link:** https://openstrate.com/agent (redirects to the same file).  
> Production deploy copies this file from the repo `SKILL.md` via `npm run build` / `build:with-sil`.

## Description

Use OpenStrate (https://openstrate.com) to reconstruct 3D geometry from video.
Upload phone, drone, or dashcam video and get back GLB/GLTF meshes, PLY point
clouds, and shareable browser viewer links — no LiDAR required.

This skill gives the agent the ability to "perceive" real-world environments in
3D by converting video footage into structured geometry the agent can reason
about, inspect, and share.

## API reference

Full docs (machine-readable and human-readable):

- **OpenAPI 3.0 spec:** https://openstrate.com/openstrate-reconstruction-api.openapi.yaml
- **Markdown with curl examples:** https://openstrate.com/openstrate-reconstruction-api.md
- **llms.txt index:** https://openstrate.com/llms.txt
- **HTML narrative docs:** https://openstrate.com/reconstruction-api/

Always consult the OpenAPI spec or Markdown docs for the latest schemas, error
codes, and field constraints before making calls.

## When to use

- The user asks to scan, reconstruct, or digitize a real-world object or scene.
- The user wants a 3D model, mesh, or point cloud from a video file.
- The user says "look at this space" / "see my room" / "create a digital twin".
- The user provides or references a .mp4 or .mov file for 3D processing.
- The user wants to view or share an interactive 3D model in the browser.

## Prerequisites

- A video file (.mp4 or .mov) of the target object or scene.
  Best results: slow, overlapping scan around the subject.
- Internet access from the gateway host.
- `curl` and `jq` available on the host (standard on macOS/Linux).

## Configuration (openclaw.json)

```jsonc
{
  "skills": {
    "openstrate": {
      "apiBaseUrl": "https://openstrate.com",  // default, rarely changed
      "apiKey": ""  // set if API requires authentication
    }
  }
}
```

If an API key is needed, set it in config or as an environment variable:

```bash
export OPENSTRATE_API_KEY="your-key-here"
```

---

## API Workflow

The reconstruction pipeline is three steps: **upload → reconstruct → poll**.
Pipeline callbacks exist internally but are NOT for integrators — always poll.

### Step 1: Upload the video

```bash
curl -X POST https://openstrate.com/api/upload \
  -F "file=@/path/to/video.mp4"
```

This returns a response with a file identifier. Extract the file/upload ID from
the response JSON for use in the next step.

### Step 2: Start reconstruction

```bash
curl -X POST https://openstrate.com/api/reconstruct \
  -H "Content-Type: application/json" \
  -d '{
    "file_id": "<file_id_from_upload>"
  }'
```

This returns a `job_id` and initial status. The reconstruction runs
asynchronously on OpenStrate's servers.

### Step 3: Poll for completion

```bash
curl https://openstrate.com/api/reconstruct/status/<job_id>
```

Poll this endpoint until `status` is `"complete"` or `"failed"`.
Recommended poll interval: **15 seconds**. Reconstruction typically takes
1–10 minutes depending on video length and complexity.

### Step 4: Retrieve artifacts

Once status is `"complete"`, the response includes artifact URLs:

| Artifact | URL pattern | Description |
|---|---|---|
| Mesh file | `/api/files/<artifact_id>` | Downloadable GLB, GLTF, or PLY |
| Preview | `/api/preview/<artifact_id>` | Thumbnail or preview image |
| Viewer | `/view/<job_id>` | Shareable interactive 3D viewer |

Download the mesh:

```bash
curl -o model.glb "https://openstrate.com/api/files/<artifact_id>"
```

Share the viewer link directly with the user:

```
https://openstrate.com/view/<job_id>
```

---

## Complete agent script

```bash
#!/bin/bash
# openstrate_reconstruct.sh — upload video, reconstruct, download result
set -euo pipefail

VIDEO_PATH="$1"
BASE_URL="${OPENSTRATE_API_BASE:-https://openstrate.com}"
OUTPUT_NAME="$(basename "${VIDEO_PATH%.*}").glb"

echo "==> Uploading $VIDEO_PATH ..."
UPLOAD_RESPONSE=$(curl -s -X POST "$BASE_URL/api/upload" \
  -F "file=@$VIDEO_PATH")
echo "Upload response: $UPLOAD_RESPONSE"

# Extract file ID (adapt jq path to match actual response schema from OpenAPI spec)
FILE_ID=$(echo "$UPLOAD_RESPONSE" | jq -r '.file_id // .id // .upload_id')
if [ "$FILE_ID" = "null" ] || [ -z "$FILE_ID" ]; then
  echo "ERROR: Could not extract file ID from upload response."
  echo "$UPLOAD_RESPONSE" | jq .
  exit 1
fi
echo "==> File uploaded: $FILE_ID"

echo "==> Starting reconstruction ..."
RECON_RESPONSE=$(curl -s -X POST "$BASE_URL/api/reconstruct" \
  -H "Content-Type: application/json" \
  -d "{\"file_id\": \"$FILE_ID\"}")
echo "Reconstruct response: $RECON_RESPONSE"

JOB_ID=$(echo "$RECON_RESPONSE" | jq -r '.job_id // .id')
if [ "$JOB_ID" = "null" ] || [ -z "$JOB_ID" ]; then
  echo "ERROR: Could not extract job ID from reconstruct response."
  echo "$RECON_RESPONSE" | jq .
  exit 1
fi
echo "==> Reconstruction started: $JOB_ID"

echo "==> Polling for completion ..."
TIMEOUT=1200  # 20 minutes max
ELAPSED=0
INTERVAL=15

while [ "$ELAPSED" -lt "$TIMEOUT" ]; do
  STATUS_RESPONSE=$(curl -s "$BASE_URL/api/reconstruct/status/$JOB_ID")
  STATE=$(echo "$STATUS_RESPONSE" | jq -r '.status')

  case "$STATE" in
    complete|completed|done)
      echo "==> Reconstruction complete!"

      # Extract artifact download URL (verify jq path against OpenAPI spec)
      GLB_URL=$(echo "$STATUS_RESPONSE" | jq -r '
        (.artifacts // .outputs // [])
        | if type == "array" then
            .[] | select(.format == "glb" or .type == "mesh") | .url
          elif type == "object" then
            .glb // .mesh // .file_url
          else empty end
      ' 2>/dev/null | head -1)

      VIEWER_URL="$BASE_URL/view/$JOB_ID"

      if [ -n "$GLB_URL" ] && [ "$GLB_URL" != "null" ]; then
        # Handle relative URLs
        if [[ "$GLB_URL" == /* ]]; then
          GLB_URL="$BASE_URL$GLB_URL"
        fi
        echo "==> Downloading mesh to $OUTPUT_NAME ..."
        curl -s -o "$OUTPUT_NAME" "$GLB_URL"
        echo "Saved: $OUTPUT_NAME ($(du -h "$OUTPUT_NAME" | cut -f1))"
      else
        echo "NOTE: Could not auto-detect GLB URL from response."
        echo "Full status response:"
        echo "$STATUS_RESPONSE" | jq .
        echo ""
        echo "Check /api/files/ paths in the response and download manually."
      fi

      echo ""
      echo "============================="
      echo "Viewer:   $VIEWER_URL"
      echo "Download: ${GLB_URL:-see status response above}"
      echo "============================="
      exit 0
      ;;

    failed|error)
      echo "ERROR: Reconstruction failed."
      echo "$STATUS_RESPONSE" | jq .
      exit 1
      ;;

    *)
      echo "  Status: $STATE — waiting ${INTERVAL}s ... (${ELAPSED}s elapsed)"
      sleep "$INTERVAL"
      ELAPSED=$((ELAPSED + INTERVAL))
      ;;
  esac
done

echo "ERROR: Timed out after ${TIMEOUT}s. Job $JOB_ID may still be running."
echo "Check manually: curl $BASE_URL/api/reconstruct/status/$JOB_ID"
exit 1
```

Usage:

```bash
bash openstrate_reconstruct.sh /path/to/video.mp4
```

---

## Example conversations

### Basic scan

```
User: "Can you turn this video into a 3D model?"
      [attaches office_scan.mp4]

Agent: I'll upload that to OpenStrate for 3D reconstruction.

      [POST /api/upload with office_scan.mp4 → file_id: "f_abc123"]
      [POST /api/reconstruct with file_id → job_id: "j_def456"]
      [GET /api/reconstruct/status/j_def456 — polls every 15s]
      ... ~3 minutes ...

Agent: Your 3D model is ready!
      - Interactive viewer: https://openstrate.com/view/j_def456
      - Downloaded: office_scan.glb (14.2 MB)

      The reconstruction looks clean. Good coverage of the desk and walls.
      The window area has minor artifacts from reflections, which is typical.
```

### Perception / spatial reasoning

```
User: "Look at my workshop and tell me roughly how much shelf space I have."
      [attaches workshop_scan.mov]

Agent: Let me reconstruct that into 3D so I can measure the space.

      [uploads → reconstructs → downloads GLB]
      [inspects mesh bounding box and vertex data]

Agent: Based on the reconstruction, your workshop is roughly 4m × 3m.
      I can see three shelf units along the back wall, each about 1.8m
      tall × 0.9m wide × 0.4m deep — approximately 5.8 m³ of total
      shelf volume, though the bottom shelves on the left unit look full.

      View the 3D model: https://openstrate.com/view/j_xyz789
```

---

## Using results downstream

Once you have artifacts, the agent can:

- **Share** the `/view/<job_id>` URL — works in any browser, no install needed.
- **Preview** via `/api/preview/<artifact_id>` for quick thumbnails.
- **Inspect geometry** using `gltf-transform info model.glb` (for GLB) or
  reading PLY headers to report vertex/face counts and bounding box.
- **Measure spaces** by extracting vertex coordinates from the mesh.
- **Feed into other tools** — 3D print slicers, Blender, game engines,
  AR viewers, or CAD software.
- **Compare scans** over time for change detection (construction progress,
  inventory monitoring, before/after).

## Tips for best scan results

- **Slow, steady movement:** Overlap coverage generously.
- **Good lighting:** Even, diffuse light minimizes artifacts.
- **Avoid reflections:** Mirrors and glass confuse reconstruction.
- **Multiple angles:** Walk around the subject; don't pan from one spot.
- **30–120 seconds** of video is usually sufficient for one object or room.
- **1080p** is plenty. 4K increases upload time without proportional gains.

## Error handling

Consult the full error table in the Markdown API docs:
https://openstrate.com/openstrate-reconstruction-api.md

Common issues:

| Symptom | Likely cause | Fix |
|---|---|---|
| Upload returns 413 | File too large | Compress or trim video |
| Reconstruct returns 400 | Invalid file_id or bad params | Check upload response; re-upload |
| Status stuck on processing | Long video or server load | Wait longer; timeout at ~20 min |
| Status returns "failed" | Insufficient overlap in video | Re-scan with more coverage |
| /api/files/ returns 404 | Job expired or not ready | Re-check status; re-run if expired |
| Connection refused | Network issue from gateway | Check internet access; verify URL |

## Important notes

- **Pipeline callbacks are NOT for integrators.** Always poll
  `GET /api/reconstruct/status/{job_id}`. Do not attempt to register
  webhook/callback endpoints.
- **Verify field names** against the OpenAPI spec before relying on jq paths.
  The script above uses fallback patterns (`// .id // .upload_id`) to handle
  minor naming variations, but the spec is authoritative.
- **Rate limits** may apply. Space out uploads if batch-processing videos.

## References

- OpenStrate: https://openstrate.com
- OpenAPI spec: https://openstrate.com/openstrate-reconstruction-api.openapi.yaml
- Markdown API docs: https://openstrate.com/openstrate-reconstruction-api.md
- HTML narrative docs: https://openstrate.com/reconstruction-api/
- llms.txt: https://openstrate.com/llms.txt
- CoMe paper (confidence-based mesh extraction): https://r4dl.github.io/CoMe/
- OpenClaw skills docs: https://docs.openclaw.ai/tools/skills