What is AgentQL?

AgentQL is a Model Context Protocol (MCP) server that enables AI agents to extract structured data from unstructured web content. It provides a bridge between AI agents and web data through API integration.

How to use AgentQL?

To use AgentQL, you need to install it via npm, set up an API key, and configure it in your preferred application that supports MCP (Model Context Protocol).

1. Install the package

   npm install -g agentql-mcp
   

2. Configure the MCP server in your application of choice (Claude, VS Code, Cursor, or Windsurf).
3. Set up your AgentQL API key in the configuration.
4. Use AgentQL tools to extract data from web pages by providing a URL and a prompt describing the data to extract.

Key features of AgentQL

• extract-web-data tool to extract structured data from a given URL using prompts.
• Easy integration with applications supporting MCP.
• One-click installation and configuration options for various IDEs and coding environments.
• Development tools and debugging support with the MCP Inspector.

Use cases of AgentQL

1. Web data extraction and structuring for analysis or machine learning models.
2. Automated content aggregation from web pages.
3. Enriching AI agent responses with real-time web data.
4. Quickly gathering lists of items (like videos, products, etc.) from web pages in a structured format.

FAQ from AgentQL

• Is AgentQL free to use?

  AgentQL requires an API key which can be obtained free of charge from their Dev Portal, but usage may be subject to rate limits or pricing tiers.

• Which applications support AgentQL?

  AgentQL can be integrated with applications that support MCP (Model Context Protocol), including Claude, VS Code, Cursor, and Windsurf.

• How can I debug issues with AgentQL?

  Use the built-in MCP Inspector by running npm run inspector to get debugging tools in your browser.

• Do I need coding skills to use AgentQL?

  Basic familiarity with command-line tools and simple JSON configuration is required. However, the one-click installation options simplify setup for beginners.

AgentQL MCP Server

This is a Model Context Protocol (MCP) server that integrates AgentQL's data extraction capabilities.

Features

Tools

• extract-web-data - extract structured data from a given 'url', using 'prompt' as a description of actual data and its fields to extract.

Installation

To use AgentQL MCP Server to extract data from web pages, you need to install it via npm, get an API key from our Dev Portal, and configure it in your favorite app that supports MCP.

Install the package

npm install -g agentql-mcp

Configure Claude

• Open Claude Desktop Settings via ⌘+, (don't confuse with Claude Account Settings)
• Go to Developer sidebar section
• Click Edit Config and open claude_desktop_config.json file
• Add agentql server inside mcpServers dictionary in the config file
• Restart the app

{
  "mcpServers": {
    "agentql": {
      "command": "npx",
      "args": ["-y", "agentql-mcp"],
      "env": {
        "AGENTQL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Read more about MCP configuration in Claude here.

Configure VS Code

For one-click installation, click one of the install buttons below:

Manual Installation

Click the install buttons at the top of this section for the quickest installation method. For manual installation, follow these steps:

Add the following JSON block to your User Settings (JSON) file in VS Code. You can do this by pressing Ctrl + Shift + P and typing Preferences: Open User Settings (JSON).

{
  "mcp": {
    "inputs": [
      {
        "type": "promptString",
        "id": "apiKey",
        "description": "AgentQL API Key",
        "password": true
      }
    ],
    "servers": {
      "agentql": {
        "command": "npx",
        "args": ["-y", "agentql-mcp"],
        "env": {
          "AGENTQL_API_KEY": "${input:apiKey}"
        }
      }
    }
  }
}

Optionally, you can add it to a file called .vscode/mcp.json in your workspace. This will allow you to share the configuration with others.

{
  "inputs": [
    {
      "type": "promptString",
      "id": "apiKey",
      "description": "AgentQL API Key",
      "password": true
    }
  ],
  "servers": {
    "agentql": {
      "command": "npx",
      "args": ["-y", "agentql-mcp"],
      "env": {
        "AGENTQL_API_KEY": "${input:apiKey}"
      }
    }
  }
}

Configure Cursor

• Open Cursor Settings
• Go to MCP > MCP Servers
• Click + Add new MCP Server
• Enter the following:
  • Name: "agentql" (or your preferred name)
  • Type: "command"
  • Command: env AGENTQL_API_KEY=YOUR_API_KEY npx -y agentql-mcp

Read more about MCP configuration in Cursor here.

Configure Windsurf

• Open Windsurf: MCP Configuration Panel
• Click Add custom server+
• Alternatively you can open ~/.codeium/windsurf/mcp_config.json directly
• Add agentql server inside mcpServers dictionary in the config file

{
  "mcpServers": {
    "agentql": {
      "command": "npx",
      "args": ["-y", "agentql-mcp"],
      "env": {
        "AGENTQL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Read more about MCP configuration in Windsurf here.

Validate MCP integration

Give your agent a task that will require extracting data from the web. For example:

Extract the list of videos from the page https://www.youtube.com/results?search_query=agentql, every video should have a title, an author name, a number of views and a url to the video. Make sure to exclude ads items. Format this as a markdown table.

[!TIP] In case your agent complains that it can't open urls or load content from the web instead of using AgentQL, try adding "use tools" or "use agentql tool" hint.

Development

Install dependencies:

npm install

Build the server:

npm run build

For development with auto-rebuild:

npm run watch

If you want to try out development version, you can use the following config instead of the default one:

{
  "mcpServers": {
    "agentql": {
      "command": "/path/to/agentql-mcp/dist/index.js",
      "env": {
        "AGENTQL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

[!NOTE] Don't forget to remove the default AgentQL MCP server config to not confuse Claude with two similar servers.

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

npm run inspector

The Inspector will provide a URL to access debugging tools in your browser.