StackEngine CMS

Tag: Claude Code MCP

  • How to Build an Agentic SEO Pipeline with Claude Code and MCP

    How to Build an Agentic SEO Pipeline with Claude Code and MCP

    An agentic SEO pipeline uses Claude Code as the orchestrator, MCP servers as the data layer, and Airtable as the storage backend. Instead of switching between keyword tools, spreadsheets, and AI chat windows, a single system runs the full research loop — from brand analysis through SERP deep dives — in one session. This post walks through the 13-step architecture behind SearchScope, part of the broader agentic SEO approach we’ve been building at StackEngine.

    Why MCP Changes the Architecture

    TL;DR: MCP lets Claude Code call external APIs as native tools, which means your AI agent can read, write, and act on live data without custom middleware.

    Model Context Protocol (MCP) turns external services into tool calls that Claude Code can invoke directly. DataForSEO becomes a keyword research tool. Airtable becomes a database. No API wrappers. No intermediate scripts. Claude Code reads the MCP tool definitions, understands what each one does, and calls them as part of its reasoning loop.

    This matters because the bottleneck in traditional SEO research isn’t the data — it’s the assembly. You pull keywords from one tool, check SERPs in another, paste results into a spreadsheet, then manually cross-reference. An MCP-based pipeline collapses that into a single orchestrated flow where Claude Code decides what to query, interprets the results, and writes structured records directly to Airtable.

    The practical difference: what used to take an afternoon of tab-switching now runs in one Claude Code session. And because the agent handles the data plumbing, you spend your time on decisions — which keywords matter, which clusters to pursue, where the gaps are.

    The 13-Step Pipeline

    TL;DR: The pipeline moves through four phases — setup, broad research, clustering, and deep dives — each building on the data from the previous phase.

    Here’s the full sequence:

    1. Verify MCP servers — Confirm DataForSEO and Airtable MCP connections are live. If either is down, the pipeline stops early instead of failing mid-run.
    2. Run brand profile — Analyze the target website, extract core topics, and determine the brand’s current positioning. This grounds every keyword decision in what the brand actually does.
    3. Expand topics into keywords — Use DataForSEO to generate keyword ideas, suggestions, and related terms for each core topic.
    4. Write keywords to Airtable — Store raw keyword data with search volume, difficulty, and intent classification. Claude Code creates the Airtable table schema through MCP if it doesn’t exist yet.
    5. Landscape analysis — Pull SERP data for priority keywords. Identify who ranks, what content types dominate, and where the gaps are.
    6. AI platform assessments — Query ChatGPT, Perplexity, Claude, and Gemini with the same prompts to see which brands and pages they cite. This is AI platform visibility tracking applied at the keyword level.
    7. Cluster results — Group keywords into semantic clusters based on SERP overlap, topic similarity, and intent alignment.
    8. Score and categorize — Rank clusters by opportunity: search volume, difficulty, AI citation gaps, and alignment with the brand’s existing content.
    9. Write results to Airtable — Store clusters with scores, categories, and priority rankings.
    10. Deep dive: full SERP analysis — For priority clusters, pull full SERP results with featured snippets, People Also Ask, and site links.
    11. Deep dive: AI platform analysis — Detailed AI visibility checks for priority terms — who gets cited, in what context, and how often.
    12. Deep dive: Google AI Mode check — Test whether Google’s AI overview triggers for priority keywords, and what content it pulls from.
    13. Write updated cluster records — Push deep dive data back to Airtable, enriching the cluster records with SERP and AI citation details.

    Steps 1–4 are setup and broad collection. Steps 5–9 are analysis and organization. Steps 10–13 are targeted deep dives on the clusters that matter most.

    How Claude Code Orchestrates the Flow

    TL;DR: Claude Code acts as the decision layer — it doesn’t just execute steps sequentially, it reads intermediate results and adjusts what happens next.

    The key architectural insight: Claude Code isn’t running a fixed script. It’s making judgment calls at each step based on what the data shows.

    After step 3, if DataForSEO returns 2,000 keyword ideas, Claude Code doesn’t blindly process all of them. It filters by relevance, deduplicates, and selects the set worth writing to Airtable. After step 5, if the landscape analysis shows a keyword cluster is dominated by massive authority sites with no realistic entry point, Claude Code deprioritizes it.

    Sub-agents handle parallelizable work. When running AI platform assessments in step 6, separate sub-agents can query each platform simultaneously instead of sequentially. This is where the architecture went through three iterations — early versions hit context window limits trying to hold all the data in a single agent’s memory. The solution: specialized sub-agents that handle discrete tasks and write results to Airtable, keeping the orchestrator’s context lean.

    The pattern looks like this:

    Orchestrator (Claude Code)
├── Brand Analyzer        → reads website, writes brand profile
├── Keyword Expander      → calls DataForSEO, writes to Airtable
├── SERP Analyzer         → pulls rankings, identifies gaps
├── AI Visibility Checker → queries 4 AI platforms in parallel
├── Cluster Engine        → groups and scores keywords
└── Deep Dive Agents      → detailed analysis on priority clusters

    Each sub-agent gets a focused task, a clear input, and writes its output to Airtable. The orchestrator reads those results and decides what runs next.

    What Three Architecture Versions Taught Us

    TL;DR: Context windows are the hard constraint. The system works when you treat Airtable as shared memory and keep each agent’s scope narrow.

    Version 1 tried to run everything in a single Claude Code session. It worked for small keyword sets. At scale, the context window filled up with raw API responses, and the agent lost track of earlier analysis.

    Version 2 introduced sub-agents but kept intermediate results in memory. Better, but still fragile — if one sub-agent returned unexpectedly large results, it could crowd out the orchestrator’s reasoning space.

    Version 3 — the current architecture — treats Airtable as the system’s shared memory. Each agent writes its results to Airtable immediately. The orchestrator reads summaries, not raw data. This mirrors how traditional keyword research breaks down at scale — the difference is that the agentic version solves the assembly problem instead of just generating more data.

    The takeaway for anyone building similar systems: design for the context window from day one. If an agent needs to hold more than a few hundred rows of data to make a decision, restructure so it doesn’t.

    Getting Started

    TL;DR: You need Claude Code, two MCP servers, and an Airtable base. The pipeline creates its own schema.

    The prerequisites are straightforward:

    • Claude Code — the orchestrator
    • DataForSEO MCP server — provides keyword data, SERP results, and AI platform queries
    • Airtable MCP server — provides structured data storage with API access
    • An Airtable base — the pipeline creates tables and fields through MCP, so you start with an empty base

    You don’t need to predefine the Airtable schema. Claude Code creates the tables it needs — keywords, clusters, SERP data, AI citations — through the Airtable MCP. This is one of the things that surprised us: the agent can design its own data model based on what it’s collecting. You review and adjust, but the initial schema is functional out of the box.

    The pipeline fits within the broader agentic SEO framework where AI agents handle research, monitoring, and analysis — and you handle the decisions about what to pursue. The hidden gem keywords that emerge from steps 7–8 are often the most valuable output: clusters with real search volume that competitors haven’t targeted because they require cross-referencing data that manual processes can’t assemble efficiently.

    Start with a single topic area. Run the pipeline end to end. Review what it finds. Then expand.

  • MCP Servers: How AIOS Connects to External Services

    MCP Servers: How AIOS Connects to External Services

    An AI Operating System needs to talk to other software. Airtable for data. WordPress for publishing. Blotato for social distribution. DataForSEO for research. The Model Context Protocol (MCP) is what makes these connections work inside an AIOS built on Claude Code — and it replaces a surprising amount of custom integration code.

    TL;DR: MCP servers give your AIOS standardized connectors to external services. Each service exposes tools that Claude Code can call directly. No SDK wrappers, no middleware, no glue code. You configure the server, and the tools appear.

    What MCP Actually Does

    TL;DR: MCP turns external APIs into callable tools inside Claude Code’s environment.

    MCP — the Model Context Protocol — is a standard for connecting AI models to external data and services. In practice, it works like this: you run an MCP server for a service (say, Airtable), and that server exposes a set of tools that Claude Code can call natively.

    The tool calls look like namespaced functions:

    mcp__airtable__list_records
mcp__airtable__create_record
mcp__airtable__update_records
mcp__airtable__search_records

    Claude Code doesn’t need to know how to authenticate with Airtable’s REST API, handle pagination, or manage rate limits. The MCP server handles all of that. Claude Code just calls the tool with the right parameters.

    This is fundamentally different from the old approach of writing API wrapper functions, managing tokens in environment variables, and building error handling for each service. MCP collapses that entire layer into a protocol.

    The Services in Content Engine AIOS

    TL;DR: Five external services power the system. Four connect via MCP. One uses a REST API directly.

    Here’s what each service does and how it connects:

    Service Role Connection Tool Prefix
    Airtable Data layer, pipeline tracking MCP server mcp__airtable__
    DataForSEO Keyword research, SERP analysis, AI visibility MCP server mcp__dfs-mcp__
    Blotato Social publishing (Twitter, LinkedIn, YouTube, TikTok) MCP server mcp__blotato__
    WordPress Website publishing (headless CMS) REST API (curl) n/a
    HeyGen AI avatar video generation REST API (curl) n/a

    Airtable is the core. Every content piece, every plan, every source record lives there. It’s the single source of truth the entire system reads from and writes to.

    DataForSEO powers the research layer — keyword data, SERP results, AI platform mentions, content analysis. Its MCP server exposes dozens of tools organized by category:

    mcp__dfs-mcp__serp_organic_live_advanced
mcp__dfs-mcp__dataforseo_labs_google_keyword_overview
mcp__dfs-mcp__ai_opt_llm_ment_search
mcp__dfs-mcp__backlinks_summary

    Blotato handles the last mile — getting approved content published to social platforms. One MCP call creates a post, another checks its status, another schedules it.

    WordPress and HeyGen connect via REST API (curl) rather than MCP. WordPress uses its native JSON API for headless publishing. HeyGen’s video generation API is called directly with curl. Both work fine — MCP isn’t required for every integration. It just makes the ones it supports cleaner.

    Why MCP Beats Custom Integrations

    TL;DR: MCP eliminates glue code, makes services interchangeable, and lets the AI handle tool selection.

    Before MCP, connecting an AI system to an external service meant writing a custom integration layer. You’d build functions to authenticate, format requests, parse responses, handle errors, and expose the results in a format the AI could use. Every service was a bespoke project.

    MCP changes three things:

    No glue code. The MCP server is the integration. You configure it once — point it at the service, provide credentials — and it exposes the full tool surface. No wrapper functions to maintain. No response parsers to debug.

    Service awareness. Claude Code can see all available MCP tools and their descriptions. It picks the right tool for the job. You don’t need to map user intent to API calls — the orchestration layer handles that.

    Standardized interface. Every MCP tool follows the same pattern: namespaced name, typed parameters, structured response. Whether you’re querying Airtable or DataForSEO, the calling convention is identical. This matters when your AIOS architecture has multiple layers that all need to interact with services.

    The practical result: adding a new service to the AIOS means configuring one MCP server, not writing an integration library.

    How MCP Servers Get Configured

    TL;DR: MCP servers are declared in a settings file. Each entry specifies the server command, environment variables, and transport.

    In Claude Code, MCP servers are configured in the project’s settings. Each server declaration looks roughly like this:

    {
  "mcpServers": {
    "airtable": {
      "command": "npx",
      "args": ["-y", "@airtable/mcp-server"],
      "env": {
        "AIRTABLE_API_KEY": "pat_xxxxx"
      }
    }
  }
}

    When Claude Code starts, it launches each configured MCP server as a subprocess. The server connects, registers its tools, and Claude Code can call them immediately.

    You can add or remove services without touching any application code. The app-in-a-folder pattern means the entire system — including its service connections — lives in a directory structure. No deployment pipeline. No infrastructure. Just configuration.

    Progressive Readiness: Services Are Optional

    TL;DR: The system works with whatever services are configured. Missing a service reduces capability but doesn’t break anything.

    This is a design principle in Content Engine AIOS, and MCP makes it easy to enforce. Every service except Airtable is optional.

    No Blotato configured? Content still gets written and saved to Airtable — you just publish manually. No DataForSEO? You skip automated keyword research but can still plan content from other sources. No HeyGen? Scripts get written but videos don’t render.

    The system degrades gracefully because each service is a discrete connector, not a dependency baked into application logic. This matters for AIOS vs traditional automation — brittle integrations are the main reason automation pipelines break. MCP’s loose coupling avoids that.

    What MCP Doesn’t Solve

    TL;DR: MCP handles tool calls, not workflows. Sequencing, validation, and error recovery still need an orchestration layer.

    MCP gives you the tools. It doesn’t tell you when to use them, in what order, or what to do when a call fails.

    In Content Engine AIOS, that’s the job of commands and agents — the orchestration layer that sequences tool calls into workflows. MCP handles “create this Airtable record.” The agent handles “create the source record first, then the content plan, then link them together” — the write order rules that keep linked records consistent.

    MCP also doesn’t handle authentication rotation, usage tracking, or cost management. You still need to manage API keys, monitor rate limits, and watch costs — especially with services like DataForSEO and HeyGen where every call has a price.

    What to Build Next

    MCP is the service layer of an AIOS. It turns external APIs into native tools. But the tools are only useful inside a system that knows when and how to use them.

    Start with the full AIOS guide for the complete architecture. If you’re building the orchestration layer, read Claude Code as an AIOS Orchestrator. For the broader system design — layers, skills, memory — see AIOS Architecture: Layers, Skills, and Memory.

  • Claude Code as an AIOS Orchestrator: Why a CLI Beats a GUI

    Claude Code as an AIOS Orchestrator: Why a CLI Beats a GUI

    Claude Code as an AIOS Orchestrator: Why a CLI Beats a GUI

    TL;DR: Claude Code works as an AIOS orchestrator because it combines file system access, tool use via MCP, reasoning capability, and extensibility through plain files. A CLI gives you transparency that GUIs hide — and when you’re building a system that makes decisions, transparency isn’t optional.

    This is part of the AIOS guide, which covers the full architecture. This article focuses on the orchestration layer — why Claude Code specifically, and why a CLI interface matters more than it seems.

    What an Orchestrator Actually Does

    TL;DR: The orchestrator is the component that reads context, decides what to do, calls tools, and produces output. In an AIOS, the AI model fills this role.

    A traditional application has a runtime that executes code. An AIOS has an orchestrator that executes judgment. It reads the project state, loads relevant context, reasons about the task, and calls external services to get things done.

    The orchestrator needs four capabilities:

    1. File system access — read instructions, write output, maintain state
    2. Tool use — call external services (databases, APIs, publishing platforms)
    3. Reasoning — make decisions based on context, not just follow a script
    4. Extensibility — gain new capabilities without rewriting the system

    Most AI interfaces give you one or two of these. A chatbot gives you reasoning but no file access. A no-code builder gives you tool use but limited reasoning. Claude Code gives you all four.

    Why Claude Code Fits

    TL;DR: Claude Code provides native file I/O, MCP tool calling, strong reasoning, and a project-folder model that aligns perfectly with the AIOS architecture.

    File system as memory. Claude Code reads and writes files as a core capability. In an AIOS, the project folder is the system’s persistent memory — CLAUDE.md for instructions, learnings/ for feedback, context/ for session state. The AI reads these files at session start and writes updates when things change. No external database needed for system state. This is the app-in-a-folder pattern — the entire system lives in one directory.

    MCP as the service layer. Model Context Protocol gives Claude Code access to external tools through a standardized interface. Each MCP server wraps a service — Airtable, WordPress, DataForSEO — and exposes named functions the AI can call. The AI doesn’t write HTTP requests. It calls mcp__airtable__create_record and gets structured results back.

    Reasoning at the core. Content strategy, editorial decisions, brand voice adaptation, deciding whether a keyword cluster is worth pursuing — these aren’t mechanical tasks. They require judgment informed by context. The AI model provides that judgment. The AIOS architecture provides the context.

    Extensibility through files. Adding a new command to an AIOS means adding a markdown file to .claude/commands/. Adding a new skill means adding a markdown file to .claude/skills/. No compilation. No deployment. No code changes. The system’s capabilities grow by adding files to a folder.

    The CLI Advantage

    TL;DR: A CLI keeps you close to the system’s actual behavior — you see what files get read, what tools get called, what decisions get made. GUIs abstract that away, which is fine until something breaks.

    The argument for GUIs is convenience. The argument for CLIs is transparency. When you’re building a system that makes decisions on your behalf, transparency wins.

    You see the work happening. In a CLI, every file read, every tool call, every decision is visible. You watch the AI load brand/profile.md, call the Airtable MCP to fetch a content plan, reason about which angle to take, and produce the draft. In a GUI, you click a button and get output. The steps in between are hidden.

    Debugging is direct. When something goes wrong in a GUI-based AI tool, you’re guessing. Was it the prompt? The context? A service timeout? In a CLI, you see exactly what the AI read, what it called, and where the process broke. The fix is usually obvious.

    The file system is the interface. An AIOS stores its state in files. A CLI works natively with files. There’s no translation layer between the system’s state and how you interact with it. Open learnings/write.md in your editor and you see exactly what the AI will read before its next write session. Try that with a GUI tool’s internal state.

    Composability. CLI tools chain together. You can pipe output, script sequences, integrate with git hooks, run commands in CI/CD pipelines. A GUI is a closed loop — you interact with it through the interface it gives you, nothing else.

    What You Give Up

    TL;DR: CLIs have a steeper learning curve and no visual dashboards. These are real tradeoffs, not dealbreakers.

    Honesty about tradeoffs matters. A CLI-first approach has real costs:

    No visual dashboard. You don’t get a drag-and-drop content calendar or a visual pipeline board. The AIOS compensates by using Airtable as the visual layer — pipeline status, content review, and approval all happen in Airtable’s interface. The CLI handles production; Airtable handles visibility.

    Learning curve. Terminal comfort is a prerequisite. If you’ve never used a CLI, the initial friction is real. But the audience for building an AIOS — AI makers — generally lives in terminals already.

    No persistent UI state. Each CLI session starts fresh in terms of interface. The AIOS compensates with the Context layer — active-work.md ensures the system knows where it left off, even if the terminal doesn’t.

    These tradeoffs are manageable because the AIOS architecture already has answers for them. The CLI handles what CLIs do best (execution, transparency, composability), and external services handle what they do best (visual interfaces, persistent state, collaboration).

    Compared to the Alternatives

    TL;DR: No-code AI builders, custom agent frameworks, and chat interfaces each solve part of the problem. None combine file access, tool use, reasoning, and extensibility the way Claude Code does.

    No-code AI builders (Zapier, Make, n8n with AI nodes) give you tool connections and visual workflow design. But the AI reasoning is limited to individual nodes — the system can’t reason across the entire workflow. And the state lives in the platform, not in files you control. For a deeper comparison, see AIOS vs Traditional Automation.

    Custom agent frameworks (LangChain, CrewAI, AutoGen) give you programmatic control and reasoning chains. But you’re writing code — Python classes, configuration objects, deployment pipelines. Every new capability requires code changes. The AIOS approach of “add a markdown file” is faster for iteration.

    Chat interfaces (ChatGPT, Claude.ai) give you reasoning but no persistent state and limited tool access. Every session starts from zero. You can’t build a system on top of a chat interface because there’s no file layer, no commands, no project structure.

    Claude Code isn’t the only possible AIOS orchestrator. But right now, the combination of native file access, MCP tool calling, strong reasoning, and the project-folder model makes it the most practical one for this architecture.

    What Makes It Work Long-Term

    TL;DR: The orchestrator choice matters less than the architecture. If the five-layer pattern is sound, the orchestrator can be swapped.

    The AIOS architecture is designed to be orchestrator-portable. The five layers — Brain, Skills, Learnings, Context, Services — are all files on disk. Any AI model with file access and tool calling capability could read them.

    Claude Code is the right choice today because it provides the best combination of the four requirements. If a better orchestrator emerges — one with larger context, faster execution, or more capable reasoning — the migration path is straightforward: point the new orchestrator at the same folder.

    That’s the real argument for the CLI approach. It’s not about Claude Code specifically. It’s about building on files and standards rather than proprietary interfaces. Files are portable. MCP is a protocol. The system survives its orchestrator.

    For the full AIOS guide — all five layers, step-by-step build instructions, and real examples — see AIOS: What an AI Operating System Is and How to Build One.