What Is Computer Use in AI? When an Agent Can Click, Type and Browse

Ask a chatbot to book a flight and you get instructions. Ask an agent with computer use and it may open the site, click through forms, and stop when payment needs your fingerprint.

That gap is why computer use became a headline feature in 2026. It is not magic. It is a loop: the model sees a screen, chooses an action, your stack runs the action, and the model sees the new screen. The product question is who runs the loop and on whose machine.

Computer use in one sentence

Computer use is tool use for graphical interfaces. Instead of calling a tidy API, the model interacts with buttons, fields, and scroll regions the way a human would, usually from repeated screenshots or a live browser view.

It shows up in developer APIs, desktop apps, and personal agents with cloud computers. The ideas rhyme, but the trust boundary moves depending on whether the browser runs in OpenAI's cloud, in your app's sandbox, or on your Mac.

OpenAI: hosted browser, your approvals

On September 29, 2026, OpenAI added computer use to the Agents API public beta. OpenAI describes agents completing tasks in an OpenAI-hosted browser, with website access approvals and sign-in handled by your application.

Read that twice. The model does not get a free pass to every URL. Your product decides allowed origins, how login works, and what gets logged. OpenAI's developer materials also treat content from webpages as untrusted, the same way you would treat pasted email or a random PDF. Prompt injection lives on pages, not only in chat.

Practical actions in the beta docs include navigation, clicking, typing, scrolling, and taking screenshots so the model can plan the next step. If you already read our OpenAI DevDay piece, treat this section as the plain-language companion, not a second keynote recap.

What builders should wire up first with OpenAI computer use

• An allowlist of site origins before the agent can click anything sensitive.
• A sign-in path you control, not a shared password sitting in the prompt.
• Human approval for irreversible steps, even if the demo skipped them.
• Logging that ties each action back to a user and a session.
• A fallback when the beta changes, because public beta means public beta.

Anthropic: your client executes the clicks

Anthropic's computer use toolset follows a different split. The model proposes structured actions. Your application executes them on a machine you operate, captures screenshots, and feeds them back.

That design keeps the keyboard and mouse under your code's governance. It also means you own compatibility, latency, and security patches on the host. Anthropic publishes which models support the computer use tools in its documentation. Check there before you assume your favorite Claude variant is in the set.

Developers often pair this pattern with a locked-down VM or a dedicated user account with few privileges. The model still only sees pixels. Your job is to make sure those pixels are not a live view of your entire corporate network.

Claude Desktop: computer use on your Mac

Claude Desktop added a computer use path aimed at people who want help inside their own apps, not only inside a browser tab. Anthropic describes it as a research preview or beta feature for Claude Pro and Max subscribers.

It ships off by default. Turning it on triggers macOS permission prompts for accessibility and screen recording, the same class of prompts other automation tools ask for. That is a feature, not a nuisance. It makes explicit that Claude may see what is on your display.

Plan eligibility matters. Some Anthropic documentation excludes Claude for Teams and Enterprise from this desktop computer use preview. If you are on a company plan, read the admin docs before you promise your team a Mac copilot.

How computer use differs from a chatbot or a single API call

A chatbot optimizes the next sentence. A REST call optimizes one structured request. Computer use optimizes a sequence of UI steps toward a goal that may take minutes and many screens.

That sequence is where cost, reliability, and safety compound. One wrong click can delete a row, send a draft, or accept a terms box. Good implementations pause before irreversible actions and keep humans on the hook for money, passwords, and public messages.

If you want a permission framework without repeating incident stories, our AI agent security guide walks through how much access to grant and where to stop. The short version: start read-only, add write access one workflow at a time, and never confuse a smooth demo with a safe default.

Jobs where computer use earns its keep

• Filling repetitive web forms when no stable API exists.
• Checking whether a staging site actually works after a deploy.
• Gathering public information from several sites into one brief you will verify.
• Driving an internal admin console that never got a modern API.

Jobs where you should stay manual or use a real API

• Payments, account recovery, or anything that moves money without a dedicated approval UI.
• Reading untrusted email or documents and then acting on instructions found inside them.
• Bulk changes in production databases because the button was visible.
• Tasks where a wrong click creates legal or reputational risk.