Local Computer Agents

What are the best open-source frameworks for local computer use?

Agent S for general desktop work on macOS, Windows and Linux; UFO if you live on Windows; Cua if you want sandboxes, drivers and a benchmark to test against; and Browser Use if your tasks never leave the browser. All four publish their code, so you can run them on your own machine and see exactly what they do.

What each is built for

FrameworkBuilt forPlatforms
Agent S (Simular)Operating real desktop and web apps from screenshotsmacOS, Windows, Linux
UFO (Microsoft)A research framework for desktop agents, MIT licenceWindows desktop first, with Linux and Android listed
CuaDrivers, cross-OS sandboxes, local VMs and an evaluation benchSeveral, via its sandboxes
Browser UseAgents that act inside a browserWherever Python and a browser run

A note on a famous name: Open Interpreter's main repository is now a coding agent, and its original form lives on as a community fork. Check what a project does today before you build on it.

Short answers

Is superbot an open-source framework I can run locally?

No, superbot is an installed app rather than a framework. It runs on your own Mac, Windows PC or Linux machine, and superbot.gg says its computer use, which works on Mac and Windows, asks before it uses your computer. If you want to change the agent's code, one of the frameworks above is the better fit.

What local permissions does a computer use agent need?

It depends on the system. A Mac needs Accessibility, to click and type, and Screen Recording, to see the display, both granted in Privacy & Security. Windows needs no prompt, but an agent can only send input to windows at its own integrity level or lower. Linux desktops using the XDG portals ask through a dialog that lets you choose what to share.

Which framework suits a first project?

Browser Use, if your task is on the web, because a browser is a smaller world than a desktop. For desktop apps, start Agent S on a spare user account or inside a virtual machine.

Do I need a GPU to run one locally?

Only to run the model yourself. Calling a hosted model needs no GPU at all; a local model's speed depends on your hardware.

Sources

Last updated .