README

A Drupal module that provides machine-optimized content browsing API endpoints for AI agents. Companion module to Commerce Agent Detector.

What It Does

AI agents visiting a Drupal site get themed HTML — full of navigation chrome, scripts, and layout markup that obscures the actual content. This module gives agents clean, structured JSON for every page on the site.

The key concept is the resolve endpoint: give it any internal URL and it returns the content as structured data, with all internal links rewritten as API URLs the agent can follow. Agents can browse the site naturally — following links, navigating menus, searching content — without ever parsing HTML.

Requirements

• PHP 8.2+
• Drupal 10.3+ or 11.x
• Commerce Agent Detector ^1.0

Installation

composer require drupal-mods/content-agent-detector
drush en content_agent_detector

API Endpoints

Discovery (Public)

Browsing (Authenticated)

How Agents Browse

1. Discover — Fetch /.well-known/agent-content.json to find all endpoints
2. Orient — Get the sitemap or menu tree to see what content exists
3. Navigate — Use the resolve endpoint to read any page
4. Follow links — Internal links in responses are rewritten as API URLs
5. Search — Use the search endpoint for keyword queries

Example: Resolve a Page

GET /api/content/agent/resolve?path=/about-us
Authorization: Bearer cad_your_token_here
Accept: application/json

{
  "status": "success",
  "content": {
    "@type": "WebPage",
    "id": 42,
    "type": "page",
    "title": "About Us",
    "url": "https://example.com/about-us",
    "api_url": "/api/content/agent/node/42",
    "body": "## Our Story\n\nWe are a company that...",
    "fields": {
      "field_subtitle": "Learn more about our team"
    },
    "links": [
      {
        "text": "Meet the Team",
        "url": "/api/content/agent/resolve?path=/team",
        "type": "internal"
      }
    ],
    "taxonomy": {
      "tags": [
        {
          "id": 5,
          "name": "Company",
          "api_url": "/api/content/agent/taxonomy/tags/5"
        }
      ]
    }
  }
}

Example: Menu Navigation

GET /api/content/agent/menu/main

{
  "status": "success",
  "menu": "main",
  "tree": [
    {
      "title": "Home",
      "api_url": "/api/content/agent/resolve?path=/",
      "weight": 0,
      "children": []
    },
    {
      "title": "Products",
      "api_url": "/api/content/agent/resolve?path=/products",
      "weight": 1,
      "children": [
        {
          "title": "New Arrivals",
          "api_url": "/api/content/agent/resolve?path=/products/new",
          "weight": 0,
          "children": []
        }
      ]
    }
  ]
}

Authentication

Uses the same authentication mechanisms as Commerce Agent Detector:

• Managed tokens — Created in the admin UI with content scope
• Bearer tokens — Authorization: Bearer cad_...
• X-Agent-Token header — X-Agent-Token: cad_...
• Legacy config tokens — From Commerce Agent Detector settings
• Agent detection — Recognized User-Agents (GPTBot, ClaudeBot, etc.)
• Anonymous readonly — Optional, for public content sites

Configuration

Navigate to Commerce → Configuration → Agent Detector → Content API tab.

Settings include:

• Allowed content types — Which node types to expose
• Allowed vocabularies — Which taxonomies to expose
• Body format — Markdown (default) or plain text
• Max body length — Truncation limit for body content
• Include media — Expose images and file attachments
• Link rewriting — Rewrite internal links as API URLs
• Anonymous readonly — Allow unauthenticated JSON requests

Content Rendering

The module converts Drupal content to clean structured data:

• Body fields — HTML converted to Markdown with heading structure, links, lists, code blocks, and tables preserved
• Entity references — Resolved to labels with API URLs
• Images — Direct file URLs with alt text and dimensions
• Files — Download URLs with MIME type and size
• Taxonomy — Grouped by vocabulary with API URLs
• Media entities — Resolved to source file URLs
• Internal links — Rewritten from paths like /about-us to API URLs like /api/content/agent/resolve?path=/about-us

Architecture

This module is a companion to Commerce Agent Detector and reuses its infrastructure:

Agent interactions with content endpoints appear on the same Commerce Agent Detector dashboard with content_ action prefixes.

Development

# Install dependencies
composer install

# Run unit tests
vendor/bin/phpunit --configuration phpunit.xml --testdox

# Check coding standards
vendor/bin/phpcs --warning-severity=0

# Run static analysis
vendor/bin/phpstan analyse --no-progress

License

GPL-2.0-or-later