Google has expanded its Gemini macOS application with a new “computer use” capability, allowing the AI to independently execute multi-step workflows across local desktop software and web browsers.
Currently rolling out in the United States, the feature enables users to automate repetitive tasks by delegating control to the AI model.
Local Application Control
The update changes direction from Gemini’s previous version—formerly called Gemini Spark—which was limited to working with local files and integrated services such as Gmail or Google Calendar, according to iTopnews.de. By using the "@Computer use" command, the AI can now move through menus, click buttons, and fill out forms inside third-party applications.
This functionality allows the model to bridge gaps between disparate software environments. Users can, for instance, extract data from local PDF documents to complete shipping forms, aggregate metrics for Google Slides, or manage batch processing tasks in specialized tools like Blender or Autodesk Fusion, as reported by iTopnews.de.
Gemini app for Mac adds computer use to control desktop apps https://t.co/IfSeG4Dene by @technacity
Background Execution Architecture
To avoid disrupting the user’s primary workspace, Gemini operates within an isolated background window. This interface provides a live preview of the AI’s actions, ensuring that the main screen, mouse cursor, and keyboard remain fully functional and under the user’s control throughout the process, ifun.de reported.
Security remains a granular component of the implementation. Users must define which applications the AI is permitted to access, and the system requires explicit, manual confirmation for sensitive actions such as payments or account logins. Each operation can be terminated instantly with a single click, providing a safety override for the automated workflow.

Regional and Subscription Constraints
Access to these features is currently restricted to English-language users in the United States who maintain an active subscription to Google AI Plus, Pro, or Ultra, according to Ifun. As noted by t3n.de, there is no confirmed timeline for when this functionality might expand to other regions, including Germany.
The development shifts the competitive landscape, as Google moves to match rivals like OpenAI’s “Dots” agents and Meta’s “Muse” concepts, while Apple continues work on its own proprietary AI agent, expected no sooner than late 2027.
The Platform Security Conflict
The timing of this rollout is strategic. As reported by t3n.de, Apple is preparing to introduce stricter authorization requirements for third-party AI agents in macOS 27.3. These updates will force users to confront persistent warning prompts whenever an external AI agent attempts to control system applications.
While Apple cites data privacy as the primary driver for these heightened security protocols, the move complicates the deployment of cross-platform agents. According to ifun.de, users who integrate these agents before the update may avoid some of these new friction points. This creates a clear incentive for Google to capture user adoption immediately.
Technical Integration Scope
The “computer use” model relies on the ability to parse visual interfaces rather than simple API hooks. By utilizing Generative AI frameworks to interpret screen data, Gemini translates natural language instructions into tactical interface movements. This architectural approach allows it to function in environments where traditional API-based automation is unavailable or unsupported by the software vendor.

The current implementation follows a pattern of “agentic” AI development, where the model manages state and context across multiple application windows. This requires much larger LLM parameters to manage the delay involved in reading the screen and making decisions in real time.
Google, founded in 1998, has evolved from its origins as a search engine into a multinational technology corporation with a diverse portfolio spanning cloud computing, consumer electronics, and artificial intelligence. The company’s influence is such that in 2003, its name entered common language as a verb defined in major dictionaries as the act of using its search engine to obtain information online.
As of November 20, 2026, the feature is limited to the specified US-based subscription tiers.