OpenClaw Web Scraper: AI Agent Structured

💡 OpenClaw Web Scraper: AI Agent Structured — Executive Summary

Technical architectural analysis, practical configuration manifests, and deployment instructions for OpenClaw Web Scraper: AI Agent Structured.

Docs

Installation

bash

curl -fsSL https://openclaw.ai/install.sh | bash
# openclaw port: http://localhost:18789/
# ollama port: http://localhost:11434/

References and Guidelines (NOTE)

Settings that appear below can be modified in ~/.openclaw/openclaw.json (restart: ollama launch openclaw)

Samples:

bash

{
  "agents": {
    "defaults": {
      "compaction": {
        "reserveTokensFloor": 4096
      },
      "model": {
        "primary": "ollama/llama3-extended:latest"
      },
      "workspace": "/Users/gimmyeonghan/.openclaw/workspace"
    }
  },
  "gateway": {
    "auth": {
      "mode": "token",
      "token": "239b95be7550ffffde7276a981f559b4c059b9290f2540d8"
    },
    "bind": "loopback",
    "controlUi": {
      "allowedOrigins": [
        "https://claw.fredric18.uk"
      ]
    },
    "mode": "local",
    "port": 18789,
    "tailscale": {
      "mode": "off",
      "resetOnExit": false
    }
  },
  "meta": {
    "lastTouchedAt": "2026-07-02T04:44:20.249Z",
    "lastTouchedVersion": "2026.6.11"
  },
  "models": {
    "mode": "merge",
    "providers": {
      "ollama": {
        "api": "ollama",
        "apiKey": "ollama-local",
        "baseUrl": "http://127.0.0.1:11434",
        "models": [
          {
            "contextWindow": 32768,
            "cost": {
              "cacheRead": 0,
              "cacheWrite": 0,
              "input": 0,
              "output": 0
            },
            "id": "llama3-extended:latest",
            "input": [
              "text"
            ],
            "name": "llama3-extended:latest",
            "params": {
              "num_ctx": 32768
            }
          },
          {
            "compat": {
              "supportsTools": true,
              "supportsUsageInStreaming": true
            },
            "contextWindow": 131072,
            "cost": {
              "cacheRead": 0,
              "cacheWrite": 0,
              "input": 0,
              "output": 0
            },
            "id": "gemma4:latest",
            "input": [
              "text"
            ],
            "maxTokens": 8192,
            "name": "gemma4:latest",
            "params": {
              "num_ctx": 131072
            },
            "reasoning": true
          }
        ]
      }
    }
  },
  "plugins": {
    "entries": {
      "ollama": {
        "enabled": true
      }
    }
  },
  "session": {
    "dmScope": "per-channel-peer"
  },
  "skills": {
    "entries": {
      "1password": {
        "enabled": false
      },
      "apple-notes": {
        "enabled": false
      },
      "apple-reminders": {
        "enabled": false
      },
      "bear-notes": {
        "enabled": false
      },
      "blogwatcher": {
        "enabled": false
      },
      "blucli": {
        "enabled": false
      },
      "camsnap": {
        "enabled": false
      },
      "clawhub": {
        "enabled": false
      },
      "coding-agent": {
        "enabled": false
      },
      "discord": {
        "enabled": false
      },
      "eightctl": {
        "enabled": false
      },
      "gemini": {
        "enabled": false
      },
      "gh-issues": {
        "enabled": false
      },
      "gifgrep": {
        "enabled": false
      },
      "github": {
        "enabled": false
      },
      "gog": {
        "enabled": false
      },
      "goplaces": {
        "enabled": false
      },
      "himalaya": {
        "enabled": false
      },
      "imsg": {
        "enabled": false
      },
      "mcporter": {
        "enabled": false
      },
      "model-usage": {
        "enabled": false
      },
      "nano-pdf": {
        "enabled": false
      },
      "obsidian": {
        "enabled": false
      },
      "openai-whisper": {
        "enabled": false
      },
      "openai-whisper-api": {
        "enabled": false
      },
      "openhue": {
        "enabled": false
      },
      "oracle": {
        "enabled": false
      },
      "ordercli": {
        "enabled": false
      },
      "peekaboo": {
        "enabled": false
      },
      "sag": {
        "enabled": false
      },
      "session-logs": {
        "enabled": false
      },
      "sherpa-onnx-tts": {
        "enabled": false
      },
      "slack": {
        "enabled": false
      },
      "songsee": {
        "enabled": false
      },
      "sonoscli": {
        "enabled": false
      },
      "spotify-player": {
        "enabled": false
      },
      "summarize": {
        "enabled": false
      },
      "things-mac": {
        "enabled": false
      },
      "tmux": {
        "enabled": false
      },
      "trello": {
        "enabled": false
      },
      "video-frames": {
        "enabled": false
      },
      "voice-call": {
        "enabled": false
      },
      "wacli": {
        "enabled": false
      },
      "xurl": {
        "enabled": false
      }
    }
  },
  "tools": {
    "profile": "coding",
    "web": {
      "search": {
        "enabled": true,
        "provider": "ollama"
      }
    }
  },
  "wizard": {
    "lastRunAt": "2026-07-02T04:02:50.527Z",
    "lastRunCommand": "doctor",
    "lastRunMode": "local",
    "lastRunVersion": "2026.6.11"
  }
}

After Cloudflare Setup…

bash

# cloudflare
sudo cloudflared service install eyJhIjoiN2U3ZG...

Allow new device access
1. Access token

bash

# 서버 실행환경에서 복제
     ✓ OpenClaw is running

       Open the Web UI:
         http://127.0.0.1:18789/#token=239b95be7550ffffde7276a981f559b4c059b9290f2540d8

Fix origin-related errors that appear when inserting tokens on the screen
1. Search for Settings>Infrastructure>Control UI Allowed Origins in the server’s UI
2. Enter your domain and save
3. Device pairing warning fix

`bash

openclaw devices list

 │
 ◇

 🦞 OpenClaw 2026.6.1 (2e08f0f) — The lobster in your shell. 🦞

 │
 ◇
 Pending (1)
 ┌──────────────────────────────────────┬─────────────────────────┬────────────────────────────────────────────────────────────────────┬────────────────────┬──────────┬─────────────┐
 │ Request                              │ Device                  │ Requested                                                          │ Approved           │ Age      │ Status      │
 ├──────────────────────────────────────┼─────────────────────────┼────────────────────────────────────────────────────────────────────┼────────────────────┼──────────┼─────────────┤
 │ 6c9d38fc-1a90-4ce6-bec1-8b5994befe09 │ c8fd6709ea8968190242dec │ roles: operator; scopes: operator.admin, operator.approvals,       │ none               │ just now │ new pairing │
 │                                      │ fdeecad90e65ade1b7bc6c3 │ operator.pairing, operator.read, operator.write                    │                    │          │             │
 │                                      │ 9458885afe7fdaddb9      │                                                                    │                    │          │             │
 └──────────────────────────────────────┴─────────────────────────┴────────────────────────────────────────────────────────────────────┴────────────────────┴──────────┴─────────────┘
 Paired (3)
 ┌────────────────────────────────────────────────────────────┬────────────┬───────────────────────────────────────────────────────────────────────────────┬────────────┬────────────┐
 │ Device                                                     │ Roles      │ Scopes                                                                        │ Tokens     │ IP         │
 ├────────────────────────────────────────────────────────────┼────────────┼───────────────────────────────────────────────────────────────────────────────┼────────────┼────────────┤
 │ 19ec60ef7576f2cc5f657b510293f50941f31642388b7bc925b4b54eaa │ operator   │ operator.admin, operator.read, operator.write, operator.approvals, operator.  │ operator   │            │
 │ 17712a                                                     │            │ pairing                                                                       │            │            │
 │ e0f7ef281c6cfaa33c65fe7b7fa014ace12f846ab275b5ae6126196c74 │ operator   │ operator.pairing                                                              │ operator   │            │
 │ dd409c                                                     │            │                                                                               │            │            │
 │ 3e489bccb92978578ee608b423ef56a6a72fe17245c7e20e9cbdafc8f5 │ operator   │ operator.admin, operator.read, operator.write, operator.approvals, operator.  │ operator   │            │
 │ c6d55c                                                     │            │ pairing                                                                       │            │            │
 └────────────────────────────────────────────────────────────┴────────────┴───────────────────────────────────────────────────────────────────────────────┴────────────┴────────────┘
 `

 openclaw devices approve 6c9d38fc-1a90-4ce6-bec1-8b5994befe09

prompt

│
     ◇

     🦞 OpenClaw 2026.6.1 (2e08f0f) — I'll refactor your busywork like it owes me money.

     │
     ◇
     │
     gateway connect failed: GatewayClientRequestError: scope upgrade pending approval (requestId: c4fdd42f-7c5f-49fb-83f6-b76f1ae63711)
     ◇
     Direct scope access failed; using local fallback.
     Approved c8fd6709ea8968190242decfdeecad90e65ade1b7bc6c39458885afe7fdaddb9 (6c9d38fc-1a90-4ce6-bec1-8b5994befe09)
     `

Model Change

bash

openclaw models set ollama/llama3
openclaw models status

Destroy

References and Guidelines (NOTE)

Since openclaw cannot be terminated, only the deletion method is specified.

bash

openclaw uninstall --all --yes
rm -rf ~/.openclaw

Reference when ollama installed together does not stop

Trouble Shooting

Compaction Reserve Token Floor

Problem: The following appears in the chat window

bash

⚠️ Auto-compaction could not recover this turn. I kept this conversation mapped to the current session. Please try again, use /compact, or use /new to start a fresh session.

To prevent this, increase your compaction buffer by setting agents.defaults.compaction.reserveTokensFloor to 20000 or higher in your config.

Solution

1. In the UI, search for Settings>AI & Agents>Compaction Reserve Token Floor
2. Set to 100000 and save
3. If this does not solve the problem, it is not a problem with Openclaw settings but a limitation of the model (Ollama, etc.)

Increase the model’s context size:

Apply

bash

ollama launch openclaw --model llama3-extended

In a prompted session
bash
```
/reset
```

4. Basically, 20000\~ 30000 is the appropriate range.

Fredric News

Leave a Reply Cancel reply

OpenClaw Web Scraper: AI Agent Structured

Docs

Installation

After Cloudflare Setup…

Model Change

Destroy

Trouble Shooting

Compaction Reserve Token Floor

Comments

Leave a Reply Cancel reply