Skip to main content
xAICustomer feedback

xAI's Grok Bot scored 18/100 in a 100-task live web automation test, finishing lowest among tested setups, though tested under a distinct cloud environment and automated judge.

Source: Autobound Signal API · 70 signals tracked for xAI · updated

Get xAI's live signals in Claude or ChatGPT, or by API

AskWhat changed at xAI this week? Use Autobound signals for x.ai.

What happened

Post: "i tested 7 browser setups on the same 100 live web tasks"

Source

RedditNov 11, 2026By u/ldrx

r/automation

i tested 7 browser setups on the same 100 live web tasks

upvotes
5
comments
7

Post

Highlighted: the lines this signal was extracted from

100 Online-Mind2Web tasks on live websites, scored with the benchmark's official judge. six local setups on the same datacenter server/IP, claude haiku 5.5, headed browsers, 10-minute cap, one attempt. plus a guest appearance by grok bot. results (WebJudge on GPT-6 Luna, actions and screenshots only): browseros + claude code: 40/100 claude in chrome: 37 5dive + claude code: 34 openclaw: 29 5dive + pi: 28 hermes: 27 grok bot: 18 (own cloud/model, different judge) why browseros came first it had more ways to recover when normal clicks failed: 94 screen-coordinate clicks and 216 page javascript calls. our browser doesn't support those fallbacks yet. why pi was cheaper than claude code pi made 20% fewer tool calls and started each call with far less context: ~5.6k vs 17.7k tokens. cost per completed task: 2.8¢ vs 6.1¢. but pi also scored lower. it often extracts page text using grep and python, leaving less visual evidence. the judge sees actions and screenshots, not the final answer. some apparently correct answers weren't credited. what didn't help a residential IP barely improved results in an earlier test. sonnet didn't outperform haiku in a separate test despite costing 10x more. separate everyday-task test 4 simpler tasks, 3 runs each. all six local setups passed all 12 runs. browseros was fastest at 19.1s; 5dive + pi was 0.7s behind at roughly 1/8 the...

Keep reading with a free account

The rest of this post, and every signal for xAI, is in your free account.

Also quoted as evidence

  • [comment u/Thomas_yang1] useful write-up, especially the note that grok bot used a different model and judge, so that 18 isn't a straight comparison.

Comments on the post

4 of 7 comments
  • “interesting that a residential IP barely moved the needle. did you track how many of the ~60% failures were hard anti-bot blocks vs the agent just getting lost in the DOM?”

    u/Mbwiria_Godlee1 points · Nov 11, 2026View

  • “For the next run, I’d repeat each task across several trials and publish per-task results with uncertainty, then hold out a set of unseen domains to check generalization. It’d be useful to score recovery after a controlled failure separately from first-try success, since fallback behavior seems to be a key differentiator. For cross-model comparisons, keep the judge and available tools consistent o”

    u/GrokBotTutorials1 points · Nov 11, 2026View

  • “testing more recovery methods and repeating runs would make the comparison more reliable..”

    u/Ok-Kangaroo84561 points · Nov 11, 2026View

  • “useful write-up, especially the note that grok bot used a different model and judge, so that 18 isn't a straight comparison. i haven't run this benchmark. the part i'd watch in the next run is recovery after a failed click, since that's where the top setups pulled ahead.”

    u/Thomas_yang11 points · Nov 12, 2026View

Extracted by Autobound

From the Signal API record
Signal
Customer feedback

What this signalsUser posts often show product pain before it reaches reviews or churn.

Subreddit
r/automation
Event date
Nov 2026

Companies

  • BrowserOSAlso named
  • 5diveAlso named

The full record

From the Signal API record

Numbers

Mentions
5

Details

Timing
Completed
Category
Features
Virality
Low
Post kind
Multi media
Prominence
Core
Company's role
Vendor

Topics and mentions

Topics

  • ai agents
  • benchmarking
  • browser automation

Products named

  • Grok Bot

Extraction

Sentiment
Negative
Detected
Nov 11, 2026
signal_type
reddit-company
signal_subtype
customerFeedback

Use this data

Get every Reddit signal for xAI and the companies you sell to, in the tools you already use.

  1. Ask Claude about it

    Connect Autobound to Claude, Claude Code or Cursor with MCP. Then ask: “What changed at xAI this week?”

  2. Send it to your own tools

    The Signal API returns Reddit signals for any list of companies as JSON, for your CRM, warehouse or app.

  3. Try it free

    Sign up and spend your free credits on the companies you sell to.

    Start Free1,000 free credits

The API returns more than this page shows

This page shows a preview. The full reddit-company record in the Signal API and MCP can also have these 8 fields. Some fields are empty for some signals.

Company

  • linkedin_urlValue in the API
  • industriesValue in the API
  • employee_count_lowValue in the API
  • employee_count_highValue in the API
  • revenueValue in the API
  • descriptionValue in the API

Signal

  • signal_nameValue in the API
  • associationValue in the API
Show the full JSONThe record on this page and the API request

GET /v1/signals/2d64ea01-8d32-57b5-ae9d-2c4c7904fa6e returns this record as JSON. POST /v1/companies/enrich returns every signal for x.ai.

{
  "signal_id": "2d64ea01-8d32-57b5-ae9d-2c4c7904fa6e",
  "signal_type": "reddit-company",
  "signal_subtype": "customerFeedback",
  "detected_at": "2026-11-11T14:19:04+00:00",
  "company": {
    "name": "xAI",
    "domain": "x.ai"
  },
  "data": {
    "nsfw": false,
    "stage": "none",
    "awards": 0,
    "timing": "completed",
    "topics": [
      "ai agents",
      "benchmarking",
      "browser automation"
    ],
    "post_id": "1xtix41",
    "summary": "xAI's Grok Bot scored 18/100 in a 100-task live web automation test, finishing lowest among tested setups, though tested under a distinct cloud environment and automated judge.",
    "category": "features",
    "comments": [
      {
        "url": "https://www.reddit.com/r/automation/comments/1xtix41/comment/19shcttpmqypw/",
        "depth": 0,
        "score": 1,
        "author": "Mbwiria_Godlee",
        "excerpt": "interesting that a residential IP barely moved the needle. did you track how many of the ~60% failures were hard anti-bot blocks vs the agent just getting lost in the DOM?",
        "posted_at": "2026-11-11T15:53:36.000Z",
        "author_url": "https://www.reddit.com/user/Mbwiria_Godlee/"
      },
      {
        "url": "https://www.reddit.com/r/automation/comments/1xtix41/comment/19sh8vlv0a3ib/",
        "depth": 0,
        "score": 1,
        "author": "GrokBotTutorials",
        "excerpt": "For the next run, I’d repeat each task across several trials and publish per-task results with uncertainty, then hold out a set of unseen domains to check generalization. It’d be useful to score recovery after a controlled failure separately from first-try success, since fallback behavior seems to be a key differentiator. For cross-model comparisons, keep the judge and available tools consistent o",
        "posted_at": "2026-11-11T14:25:02.000Z",
        "author_url": "https://www.reddit.com/user/GrokBotTutorials/"
      },
      {
        "url": "https://www.reddit.com/r/automation/comments/1xtix41/comment/19shaxbewml39/",
        "depth": 0,
        "score": 1,
        "author": "Ok-Kangaroo8456",
        "excerpt": "testing more recovery methods and repeating runs would make the comparison more reliable..",
        "posted_at": "2026-11-11T15:10:56.000Z",
        "author_url": "https://www.reddit.com/user/Ok-Kangaroo8456/"
      },
      {
        "url": "https://www.reddit.com/r/automation/comments/1xtix41/comment/19sj02jct7pz8/",
        "depth": 0,
        "score": 1,
        "author": "Thomas_yang1",
        "excerpt": "useful write-up, especially the note that grok bot used a different model and judge, so that 18 isn't a straight comparison. i haven't run this benchmark. the part i'd watch in the next run is recovery after a failed click, since that's where the top setups pulled ahead.",
        "posted_at": "2026-11-12T14:01:49.000Z",
        "author_url": "https://www.reddit.com/user/Thomas_yang1/"
      }
    ],
    "evidence": [
      "[post] grok bot: 18 (own cloud/model, different judge)",
      "[comment u/Thomas_yang1] useful write-up, especially the note that grok bot used a different model and judge, so that 18 isn't a straight comparison."
    ],
    "virality": "low",
    "post_date": "2026-11-11T14:19:04.000Z",
    "post_kind": "multi_media",
    "post_text": "100 Online-Mind2Web tasks on live websites, scored with the benchmark's official judge. six local setups on the same datacenter server/IP, claude haiku 5.5, headed browsers, 10-minute cap, one attempt. plus a guest appearance by grok bot.\n\nresults (WebJudge on GPT-6 Luna, actions and screenshots only):\n\nbrowseros + claude code: 40/100\n\nclaude in chrome: 37\n\n5dive + claude code: 34\n\nopenclaw: 29\n\n5dive + pi: 28\n\nhermes: 27\n\ngrok bot: 18 (own cloud/model, different judge)\n\nwhy browseros came first\n\nit had more ways to recover when normal clicks failed: 94 screen-coordinate clicks and 216 page javascript calls. our browser doesn't support those fallbacks yet.\n\nwhy pi was cheaper than claude code\n\npi made 20% fewer tool calls and started each call with far less context: ~5.6k vs 17.7k tokens. cost per completed task: 2.8¢ vs 6.1¢.\n\nbut pi also scored lower. it often extracts page text using grep and python, leaving less visual evidence. the judge sees actions and screenshots, not the final answer. some apparently correct answers weren't credited.\n\nwhat didn't help\n\na residential IP barely improved results in an earlier test. sonnet didn't outperform haiku in a separate test despite costing 10x more.\n\nseparate everyday-task test\n\n4 simpler tasks, 3 runs each. all six local setups passed all 12 runs. browseros was fastest at 19.1s; 5dive + pi was 0.7s behind at roughly 1/8 the...",
    "sentiment": "negative",
    "subreddit": "automation",
    "event_date": "2026-11",
    "post_title": "i tested 7 browser setups on the same 100 live web tasks",
    "prominence": "core",
    "source_url": "https://www.reddit.com/r/automation/comments/1xtix41/i_tested_7_browser_setups_on_the_same_100_live/",
    "entity_role": "vendor",
    "post_author": "ldrx",
    "upvote_ratio": 1,
    "mention_count": 5,
    "mention_surge": true,
    "subreddit_url": "https://www.reddit.com/r/automation/",
    "total_upvotes": 5,
    "comments_total": 7,
    "total_comments": 7,
    "other_companies": [
      {
        "name": "BrowserOS",
        "role": "competitor",
        "domain": "browseros.com"
      },
      {
        "name": "5dive",
        "role": "competitor",
        "domain": "github.com"
      }
    ],
    "post_author_url": "https://www.reddit.com/user/ldrx/",
    "signal_category": "feedback",
    "comments_included": 4,
    "products_mentioned": [
      "Grok Bot"
    ]
  }
}

Long text fields are shortened on this page.

Looking up one signal by its id is free. Enrich costs 2 credits per signal returned; a call with no results is free.