Browser

Browser#

gptme includes a browser tool that lets the assistant load pages, read their content, take screenshots, and interact with web pages.

Tools to let the assistant control a browser, including:

  • loading pages

  • reading their contents

  • searching the web

  • taking screenshots (Playwright only)

  • getting ARIA accessibility snapshots (Playwright only)

  • interactive browsing: click, fill forms, scroll (Playwright only)

  • reading PDFs (with page limits and vision fallback hints)

  • converting PDFs to images (using pdftoppm, ImageMagick, or vips)

Two backends are available:

Playwright backend:

  • Full browser automation with screenshots

  • Installation:

    pipx install 'gptme[browser]'
# We need to use the same version of Playwright as the one installed by gptme
# when downloading the browser binaries. gptme will attempt this automatically
PW_VERSION=$(pipx runpip gptme show playwright | grep Version | cut -d' ' -f2)
pipx run playwright==$PW_VERSION install chromium-headless-shell

  • To use Firefox instead of Chromium (useful for pages that block headless Chromium):

    pipx run playwright==$PW_VERSION install firefox
export GPTME_BROWSER_ENGINE=firefox

  • To use a custom browser executable (e.g. a fingerprint-patched Firefox build such as Camoufox):

    # Absolute or relative filesystem path
export GPTME_BROWSER_ENGINE=/usr/local/bin/camoufox

# Executable name on PATH (resolved via which)
export GPTME_BROWSER_ENGINE=camoufox

    Custom executables are launched with the Firefox engine so they receive the same Playwright context options. This is the recommended approach for evading bot-detection on pages that fingerprint headless browsers.

  • To use an existing Chromium-compatible browser over Chrome DevTools Protocol instead of launching Playwright’s bundled Chromium, start it with remote debugging enabled and set GPTME_BROWSER_CDP_URL:

    chromium --remote-debugging-port=9222
export GPTME_BROWSER_CDP_URL=http://127.0.0.1:9222

Lynx backend:

  • Text-only browser for basic page reading and searching

  • No screenshot support

  • Installation:

    # On Ubuntu
sudo apt install lynx
# On macOS
brew install lynx
# or any other way that gets you the `lynx` command

Provider Native Search:

  • When using Anthropic Claude models, native web search can be enabled

  • This uses Anthropic’s built-in web search instead of web scraping

  • More reliable than Google/DuckDuckGo scraping (which is blocked by bot detection)

  • Configuration:

    export GPTME_ANTHROPIC_WEB_SEARCH=true
export GPTME_ANTHROPIC_WEB_SEARCH_MAX_USES=5  # Optional, default is 5

Note

This is an experimental feature. It needs some work to be more robust and useful.

Instructions

### When to use browser

Use browser to fetch live web content, search the web, interact with pages, or
take screenshots. Prefer browser over memory for current information. For
services with APIs, prefer shell or Python over scraping.

Examples

Reading docs

User 
how does gptme work?
Assistant 
Let's read the docs.
ipython


read_url('https://gptme.org/docs/')

Answer question from URL with browsing

User 
find out which is the latest ActivityWatch version from superuserlabs.org
Assistant 
Let's browse the site.
ipython


read_url('https://superuserlabs.org/')
System 
`https://superuserlabs.org/


... [ActivityWatch](https://activitywatch.net/) ...

`
Assistant 
Couldn't find the answer on the page. Following link to the ActivityWatch website.
ipython


read_url('https://activitywatch.net/')
System 
`https://activitywatch.net/


... Download latest version v0.12.2 ...

`
Assistant 
The latest version of ActivityWatch is v0.12.2

Searching

User 
who is the founder of ActivityWatch?
Assistant 
Let's search for that.
ipython


search('ActivityWatch founder')
System 
`result


ActivityWatch was founded by Erik Bjäreholt in 2016...

`
Assistant 
The founder of ActivityWatch is Erik Bjäreholt.

Searching for latest information

User 
what are the latest developments in AI?
Assistant 
Let me search for that.
ipython


search('latest developments in AI')
System 
`result


Based on recent developments, AI has seen significant advances...

`
Assistant 
Based on the search results, here are the latest AI developments...

Take screenshot of page

User 
take a screenshot of the ActivityWatch website
Assistant 
Certainly! I'll use the browser tool to screenshot the ActivityWatch website.
ipython


screenshot_url('https://activitywatch.net')
System 
`result


Screenshot saved to screenshot.png

`

Get ARIA snapshot to see interactive elements

User 
what interactive elements are on example.com?
Assistant 
Let me get the accessibility snapshot of the page.
ipython


snapshot_url('https://example.com')
System 
`result


Page: Example Domain
URL: https://example.com/

- WebArea "Example Domain":
  - heading "Example Domain" [level=1]
  - text "This domain is for use in illustrative examples..."
  - link "More information..."

`
Assistant 
The page has a heading "Example Domain", a paragraph with description text, and a link "More information...".

Interactive browsing: open page, click, fill

User 
search for gptme on example.com's search form
Assistant 
I'll open the page, fill the search form, and click submit.
ipython


open_page('https://example.com')
System 
`result


Page: Example
URL: https://example.com/

- WebArea "Example":
  - textbox "Search" [name="q"]
  - button "Go"

`
Assistant 
I can see a search box and button. Let me fill in the search and click Go.
ipython


fill_element('input[name="q"]', 'gptme')
System 
`result


Page: Example
URL: https://example.com/

- WebArea "Example":
  - textbox "Search" [name="q"]: gptme
  - button "Go"

`
ipython


click_element('text=Go')
System 
`result


Page: Search Results
URL: https://example.com/search?q=gptme

- WebArea "Search Results":
  - heading "Results for: gptme"
  - link "gptme on GitHub"

`
Assistant 
The search was submitted and the page now shows results for "gptme".

Read full text content of interactive page

User 
what does the article say?
Assistant 
Let me read the full text content of the current page.
ipython


read_page_text()
System 
`result


# Article Title

The article discusses...

`
Assistant 
The article covers [summary of content].

Read URL and check browser logs

User 
read this page and check if there are any console errors
Assistant 
I'll read the page first and then check the browser logs.
ipython


read_url('https://example.com')
System 
`https://example.com


This domain is for use in illustrative examples...

`
Assistant 
Now let me check the browser console logs:
ipython


read_logs()
System 
`result


No logs or errors captured.

`

Read PDF document

User 
read this research paper from arxiv
Assistant 
I'll read the PDF and extract its text content.
ipython


read_url('https://arxiv.org/pdf/2410.12361v2')
System 
`result


--- Page 1 ---
[PDF text content...]

--- Page 2 ---
[More content...]

---
**Note**: This PDF has 42 pages. Showing first 10 pages.
To read more pages, use: `read_url('...', max_pages=N)` where N is the desired count, or 0 for all pages.

**Tip**: If this text extraction seems incomplete or garbled (common with scanned documents, complex layouts, or image-heavy PDFs), try vision-based reading: convert pages to images using a PDF-to-image tool, then use the vision tool to analyze them.

`
Assistant 
I've extracted the text from the PDF. The paper discusses [summary of key points]...
gptme.tools.browser.click_element(selector: str) → str

Click an element on the current page and return updated ARIA snapshot.

Requires open_page() to be called first.

Parameters:

selector –

Playwright selector to find the element. Supports:

  • CSS: “#submit-btn”, “.nav-link”, “button”

  • Text: “text=Submit”, “text=Log in”

  • Role: “role=button[name=’Submit’]”

  • Chained: “form >> text=Submit”

gptme.tools.browser.close_page() → str

Close the current interactive browsing page.

Frees browser resources. A new page can be opened with open_page().

gptme.tools.browser.fill_element(selector: str, value: str) → str

Fill a form field on the current page and return updated ARIA snapshot.

Requires open_page() to be called first. Clears any existing value before filling.

Parameters:

  • selector – Playwright selector for the input/textarea element.

  • value – Text to fill into the field.

gptme.tools.browser.get_current_url() → str

Return the URL of the currently open browser page.

Useful after a redirect, navigation, or login flow to confirm where the browser ended up.

Returns:

The current URL as a string.

Raises:

RuntimeError – If no page is currently open.

Example:

open_page("https://example.com/login")
fill_element("#username", "alice")
click_element("text=Log in")
url = get_current_url()   # confirm redirect to /dashboard
gptme.tools.browser.has_lynx() → bool

Check if lynx is available.

gptme.tools.browser.has_playwright() → bool

Check if playwright is available.

gptme.tools.browser.hover_element(selector: str) → str

Hover over an element on the current page and return updated ARIA snapshot.

Triggers mouseover/mouseenter events, revealing hover-only content such as dropdown menus, tooltips, and contextual buttons. Use before clicking a menu item that only appears on hover.

Requires open_page() to be called first.

Parameters:

selector – Playwright selector for the element to hover over.

Returns:

Updated ARIA snapshot of the page after the hover.

Example:

open_page("https://example.com")
hover_element("text=Products")   # reveal dropdown
click_element("text=Pricing")    # click item that appeared
gptme.tools.browser.load_browser_state(path: str) → str

Load a previously saved browser session (cookies, localStorage) from a file.

In-session complement to save_browser_state(). Restores authentication state without requiring a process restart or setting the GPTME_BROWSER_STORAGE_STATE environment variable.

After calling this, call open_page(url) to start a browser session with the restored cookies and localStorage.

Typical workflow:

# First session — log in and save state:
open_page("https://x.com/login")
fill_element("#username", "you@example.com")
fill_element("#password", "hunter2")
click_element("text=Log in")
save_browser_state("~/.config/gptme/twitter-session.json")

# Later in the same session (or a new one):
load_browser_state("~/.config/gptme/twitter-session.json")
open_page("https://x.com")           # opens already logged in
click_element("text=What is happening?!")
fill_element('[data-testid="tweetTextarea_0"]', "hello from gptme!")
click_element('[data-testid="tweetButtonInline"]')
Parameters:

path – Path to the session JSON previously written by save_browser_state(). ~ is expanded to the home directory.

Returns:

Confirmation string. The next open_page() will use the restored state.

Raises:

FileNotFoundError – If path does not exist.

gptme.tools.browser.open_page(url: str) → str

Open a page for interactive browsing. Returns ARIA accessibility snapshot.

Use this instead of read_url() when you need to interact with the page (click buttons, fill forms, scroll). The page stays open for subsequent click_element(), fill_element(), and scroll_page() calls.

The output includes a metadata header with the page title and current URL.

gptme.tools.browser.pdf_to_images(url_or_path: str, output_dir: str | Path | None = None, pages: tuple[int, int] | None = None, dpi: int = 150) → list[Path]

Convert PDF pages to images using auto-detected CLI tools.

Auto-detects and uses the first available tool: pdftoppm, ImageMagick convert, or vips.

Parameters:

  • url_or_path – URL or local path to PDF file

  • output_dir – Directory to save images (default: creates temp directory)

  • pages – Optional tuple of (first_page, last_page) to convert (1-indexed). If None, converts all pages.

  • dpi – Resolution for output images (default: 150)

Returns:

List of paths to generated PNG images

Raises:

Example

>>> images = pdf_to_images("https://example.com/doc.pdf")
>>> for img in images:
...     view_image(img)  # Analyze with vision tool
gptme.tools.browser.press_key(key: str) → str

Press a keyboard key or shortcut in the current browser page.

Dispatches the key event to the focused element (or document). Use for submitting forms (Enter), navigating menus (ArrowDown), dismissing dialogs (Escape), or triggering shortcuts (e.g. Control+a).

Requires open_page() to be called first.

Parameters:

key – Playwright key name. Examples: "Enter", "Tab", "Escape", "ArrowDown", "Control+a", "Meta+k".

Returns:

Updated ARIA snapshot of the page after the key press.

Example:

open_page("https://example.com/search")
fill_element("[name='q']", "gptme")
press_key("Enter")
gptme.tools.browser.read_logs() → str

Read browser console logs from the last read URL.

gptme.tools.browser.read_page_text() → str

Read the full text content of the current interactive page as Markdown.

Requires open_page() to be called first. Returns the page body converted to Markdown, preserving text formatting. Useful for reading article text, documentation, or other content after navigating to a page.

Unlike read_url(), this reads from the current interactive session — so it reflects the page state after any clicks, form fills, or navigation.

gptme.tools.browser.read_url(url: str, max_pages: int | None = None) → str

Read a webpage or PDF in a text format.

Parameters:

  • url – URL to read

  • max_pages – For PDFs only - maximum pages to read (default: 10). Set to 0 to read all pages. Ignored for web pages.

gptme.tools.browser.save_browser_state(path: str) → str

Save the current browser session (cookies, localStorage) to a file.

Captures the full authentication state of the active browser context so it can be restored in a future session via GPTME_BROWSER_STORAGE_STATE.

Call this after logging in to a site with open_page() + fill_element() + click_element() so you don’t have to re-authenticate next time.

Typical workflow:

open_page("https://x.com/login")
fill_element("#username", "you@example.com")
fill_element("#password", "hunter2")
click_element("text=Log in")
save_browser_state("~/.config/gptme/twitter-session.json")
# Next session: export GPTME_BROWSER_STORAGE_STATE=~/.config/gptme/twitter-session.json
Parameters:

path – File path to write the session JSON. Directories are created automatically. ~ is expanded to the home directory.

Returns:

Confirmation string with the absolute path where the state was saved.

gptme.tools.browser.screenshot_url(url: str, path: Path | str | None = None) → Path

Take a screenshot of a webpage.

gptme.tools.browser.scroll_page(direction: str = 'down', amount: int = 500) → str

Scroll the current page and return updated ARIA snapshot.

Requires open_page() to be called first.

Parameters:

  • direction – “up” or “down” (default: “down”)

  • amount – Pixels to scroll (default: 500)

gptme.tools.browser.search(query: str, engine: Literal['google', 'duckduckgo', 'perplexity'] | None = None) → str

Search for a query on a search engine.

If no engine is specified, automatically chooses the best available backend and falls back to the next usable backend on failure.

gptme.tools.browser.search_playwright(query: str, engine: Literal['google', 'duckduckgo', 'perplexity'] = 'google') → str

Search for a query on a search engine using Playwright.

gptme.tools.browser.select_option(selector: str, value: str) → str

Select an option from a <select> dropdown on the current page.

Requires open_page() to be called first.

Parameters:

  • selector – Playwright selector for the <select> element.

  • value – The option value attribute or visible text to select.

Returns:

Updated ARIA snapshot of the page after the selection.

Example:

open_page("https://example.com/order")
select_option("[name='size']", "large")
click_element("text=Add to cart")
gptme.tools.browser.snapshot_page() → str

Get the ARIA accessibility snapshot of the current interactive page.

Returns the structured accessibility tree of the page open via open_page(), reflecting all DOM changes made by subsequent interactions. Use to re-read the current page state without triggering any action.

Returns:

Structured ARIA snapshot including page title and current URL.

Raises:

RuntimeError – If no page is currently open.

Example:

open_page("https://example.com/form")
fill_element("[name='email']", "user@example.com")
state = snapshot_page()   # verify the field was filled before submitting
click_element("text=Submit")
gptme.tools.browser.snapshot_url(url: str) → str

Get the ARIA accessibility snapshot of a webpage.

Returns a structured text representation of the page’s accessibility tree, showing interactive elements (buttons, links, inputs) with their roles and names. Useful for understanding page structure and finding elements to interact with.

The output includes a metadata header with the page title and current URL (which may differ from the requested URL after redirects).

gptme.tools.browser.wait_for_element(selector: str, timeout_ms: int = 5000) → str

Wait for a DOM element to become visible on the current page.

Blocks until the element matching selector is visible, then returns the updated ARIA snapshot. Use after actions that trigger dynamic content loading (modals, async renders, redirects).

Requires open_page() to be called first.

Parameters:

  • selector – Playwright selector for the element to wait for.

  • timeout_ms – Maximum wait time in milliseconds (default: 5000).

Returns:

Updated ARIA snapshot once the element is visible.

Example:

open_page("https://x.com/compose/tweet")
wait_for_element("[data-testid='tweetTextarea_0']", timeout_ms=8000)
fill_element("[data-testid='tweetTextarea_0']", "Hello from gptme!")
click_element("[data-testid='tweetButtonInline']")

Backends#

Lynx#

Text-only fallback for basic page reading and web search. No screenshot support.

# Ubuntu / Debian
sudo apt install lynx
# macOS
brew install lynx

Engine configuration (GPTME_BROWSER_ENGINE)#

The Playwright backend accepts these forms for GPTME_BROWSER_ENGINE:

Value

Meaning

chromium (default)

Playwright’s bundled Chromium headless shell

firefox

Playwright’s bundled Firefox

/path/to/binary

Custom executable at a filesystem path

executable-name

Executable resolved via $PATH (shutil.which)

Custom executables (path or name) are always launched with the Firefox engine so they receive the same Playwright browser-context options.

Use Firefox instead of Chromium#

Useful for pages that detect and block headless Chromium.

PW_VERSION=$(pipx runpip gptme show playwright | grep Version | cut -d' ' -f2)
pipx run playwright==$PW_VERSION install firefox
export GPTME_BROWSER_ENGINE=firefox
gptme "read https://example.com"

Use a fingerprint-patched Firefox (anti-detection)#

Some pages fingerprint headless browsers even when Firefox is used. A patched build such as Camoufox or invisible-playwright is a drop-in replacement that passes most fingerprint checks.

By absolute path:

export GPTME_BROWSER_ENGINE=/usr/local/bin/camoufox-runner
gptme "read https://bot-detection-test.vercel.app"

By executable name on $PATH:

# Assuming camoufox is on your PATH
export GPTME_BROWSER_ENGINE=camoufox
gptme "screenshot https://example.com"

gptme detects that the value is not a named engine, resolves it via shutil.which, and passes it to Playwright’s executable_path= kwarg when launching Firefox.

Example: Camoufox setup

# Install Camoufox (fingerprint-patched Firefox)
pip install camoufox
python -m camoufox fetch           # downloads the patched Firefox binary

# Point gptme at it
export GPTME_BROWSER_ENGINE=$(python -m camoufox path)
gptme "read https://example.com"

CDP mode (GPTME_BROWSER_CDP_URL)#

Connect to an already-running Chromium-based browser over the Chrome DevTools Protocol instead of launching a new one. Useful when you want to reuse an existing authenticated browser session.

# Start Chrome/Chromium with remote debugging
chromium --remote-debugging-port=9222

# Connect gptme to it
export GPTME_BROWSER_CDP_URL=http://127.0.0.1:9222
gptme "read https://example.com"

Note

CDP only works with Chromium-based browsers. GPTME_BROWSER_ENGINE is ignored in CDP mode.

Session persistence (GPTME_BROWSER_STORAGE_STATE)#

Save login cookies and local storage so authenticated sessions persist across restarts.

# Log in once and save state
gptme "open https://x.com/login and log in with user@example.com / hunter2, then save_browser_state ~/.config/gptme/twitter.json"

# Reuse in the next session
export GPTME_BROWSER_STORAGE_STATE=~/.config/gptme/twitter.json
gptme "tweet 'hello from gptme'"

Environment variables#

Variable

Default

Description

GPTME_BROWSER_ENGINE

chromium

Engine or executable: chromium, firefox, a path, or a name on $PATH

GPTME_BROWSER_CDP_URL

(unset)

WebSocket URL of an existing Chrome DevTools Protocol server

GPTME_BROWSER_STORAGE_STATE

(unset)

Path to a Playwright storage-state JSON file for persistent sessions

FAQ#

Does the browser tool bypass CAPTCHAs?

No. The Playwright backend is a real browser engine (headless Chromium or Firefox), so it behaves the same as any headless browser — some CAPTCHAs will block it. gptme does not currently expose a headed-mode toggle for the built-in Playwright launcher. To improve success on sites that detect headless Chromium, try Firefox:

pipx run playwright==$PW_VERSION install firefox
export GPTME_BROWSER_ENGINE=firefox

You can also connect to an existing Chromium-compatible browser over Chrome DevTools Protocol:

chromium --remote-debugging-port=9222
export GPTME_BROWSER_CDP_URL=http://127.0.0.1:9222

Can I use a full GUI browser with extensions?

Yes — via the How to Automate GUIs with Computer Use Docker image, which runs a real Chromium browser inside a VNC-accessible desktop. Extensions, GUI interaction, and anything that needs a visible browser window all work there. See the Computer tool and How to Automate GUIs with Computer Use for setup details.

Can I run the browser tool inside Docker?

The standard Playwright backend works in Docker (headless mode, no display required). For headed/GUI mode inside Docker, use the computer-use Docker image which bundles a VNC server and a full desktop environment. See How to Automate GUIs with Computer Use for details.

The page is blocking my scrape — what should I try?

In order:

  1. Switch backends: GPTME_BROWSER_ENGINE=firefox (different fingerprint than Chromium)

  2. Connect to an existing Chromium browser: GPTME_BROWSER_CDP_URL=http://127.0.0.1:9222

  3. Use Anthropic native search (Claude models only): GPTME_ANTHROPIC_WEB_SEARCH=true

  4. Use the Computer tool with the VNC Docker image for full GUI browser control

Contents