GitHub Copilot Computer Use Tutorial: Control Desktop Apps on Windows and Mac
GitHub Copilot can now control desktop apps on Windows and macOS. This tutorial shows how to enable computer use, manage permissions, and test safe GUI automation.
On this page
GitHub Copilot can now control desktop applications instead of stopping at the terminal, files, or application programming interfaces. The new GitHub Copilot computer use feature entered public preview on October 1, 2026, in Copilot CLI and the GitHub Copilot app for Windows and macOS. It can read accessible application content and visual context, click controls, type, press keys, scroll, drag items, and move through workflows across applications, which makes it useful for software that has no API, command-line interface, or Model Context Protocol integration.Ā
What GitHub Copilot computer use actually changes
Until now, an AI coding assistant could often work with a terminal command or a structured tool, but many desktop programs expose neither. Computer use fills that gap by allowing Copilot to operate the graphical interface itself. That matters for legacy business applications, desktop utilities, presentation software, and other programs where a developer or tester normally has to perform a sequence of mouse and keyboard actions manually. GitHub recommends using a more direct tool such as an API, terminal command, filesystem tool, dedicated browser tool, or Model Context Protocol server when one is available because those interfaces generally provide more predictable structured data.
The feature is currently a public preview, so its behavior can change. It is also different from giving an agent unrestricted control of the machine: computer use is disabled by default, follows the permission settings of the Copilot surface, and can be interrupted while it is working. On macOS, the operating system also requires Accessibility and Screen Recording permissions for the relevant interactions.Ā
Check the supported setup before enabling it
Computer use is available for local sessions on Windows and macOS in GitHub Copilot CLI and the GitHub Copilot app. GitHub's documentation says Copilot CLI is available with all Copilot plans, although organization-managed policies can prevent an organization user from using the feature. The same documentation identifies computer use as a public-preview capability, so the interface and requirements should be treated as changeable rather than as a permanent release contract.Ā
If you are using macOS, have your Accessibility and Screen Recording permissions ready. Accessibility allows Copilot's computer-use tooling to interact with application controls, while Screen Recording allows it to inspect application windows when visual context is required. Windows does not have those two macOS permission steps in the documented setup. Before testing an important application, start with a harmless task that only reads information so you can verify the complete setup without changing data.Ā
Enable computer use in GitHub Copilot CLI
Open an interactive GitHub Copilot CLI session and first check whether computer use is already available. The status command reports the state of the bundled computer-use plugin, its Model Context Protocol server, and associated skills. This gives you a simple checkpoint before asking Copilot to control an application.
/computer show
If the feature is available, enable it with the following command. GitHub says this saves your preference and enables the bundled computer-use plugin.
/computer on
Run the status command again after enabling it.
/computer show
The useful result at this stage is that the computer-use components report as enabled. If your organization has disabled computer use through managed settings, enabling it locally cannot override that policy. If you later want to turn the feature off, use the corresponding disable command.
/computer off
These commands are the documented control surface for computer use in Copilot CLI: /computer show checks the state, /computer on enables it, and /computer off disables it.Ā
Enable computer use in the GitHub Copilot app
If you prefer the desktop application rather than the CLI, open the GitHub Copilot app settings and select Computer Use. Turn on Enable Computer Use. On macOS, the same screen guides you through granting Accessibility and Screen Recording permissions, and the documentation provides a Check again action if the application does not immediately detect a permission change.Ā
You can also use /computer on from a session in the app. To inspect the status, use /computer show. If the Computer Use settings page is missing, GitHub recommends checking whether the account and operating system are supported rather than assuming the feature itself is broken.Ā
Give Copilot a task it can safely verify
The first useful test should be read-only. Instead of asking Copilot to delete, submit, purchase, publish, or modify something, tell it to open a known desktop application and report information visible in its main window. GitHub's own example follows this pattern by asking Copilot to open an application and summarize its status without changing values or submitting forms. The important part is the combination of the intended outcome, the application involved, and the constraints that define what Copilot must not do.Ā
Open APP_NAME and summarize the status information shown in the
main window. Do not change any values or submit any forms.
Replace APP_NAME with an application installed on your computer. If the task succeeds, Copilot should be able to inspect the application and return the requested information without submitting or changing anything. This is a better first test than immediately asking it to complete a multi-step transaction because you can verify that the application is being interpreted correctly before allowing modifications.
Understand the approval prompt before allowing actions
Computer use follows the active tool-permission settings, which determine whether Copilot asks before controlling an application. When approval is required, GitHub documents choices for allowing access for the current session, saving an application as always allowed, or declining the request. A saved approval applies to the GitHub Copilot CLI and the GitHub Copilot app on the same computer, while configured deny rules take precedence.Ā
For development and testing, temporary approval is easier to reason about than permanent approval. If an application contains financial information, customer records, passwords, production controls, or other sensitive data, inspect the requested application and action before granting access. GitHub specifically warns that changing window states and unexpected on-screen content can cause computer use to select the wrong control, enter text in the wrong place, repeat an action, or stop unexpectedly.Ā
If Copilot starts doing something you did not intend, stop it immediately. In Copilot CLI, press Esc twice; in the Copilot app, use Stop or press Esc. A stop action interrupts the active operation, but it should not replace reviewing the task before granting permission in the first place.Ā
Use computer use where APIs and CLIs are missing
The strongest programming-related use case is software that developers still have to operate through a graphical interface. Imagine a legacy desktop tool that can display an application's status but provides no API and no command-line interface. A conventional automation script has little structured access to that program, while computer use can potentially read the visible interface and perform the same controls a person would use. GitHub explicitly positions the feature for legacy and GUI-only software that lacks an API, command-line interface, or Model Context Protocol integration.Ā
This also creates a useful boundary for deciding when not to use it. If a database has a supported command-line client, use the command-line client; if a service exposes an API, use the API; and if a compatible Model Context Protocol server exposes the required operation, that structured tool is generally easier to verify. Computer use is most valuable when the graphical interface is genuinely the only practical route. GitHub's documentation makes the same distinction, noting that direct tools generally provide more structured information and predictable results.Ā
Test computer use without turning it into blind automation
Computer use should be treated as supervised automation rather than a deterministic test runner. A graphical interface can change because of a different window size, application update, dialog box, timing difference, or unexpected notification, and those changes can alter what the agent sees. GitHub warns that computer use may select the wrong control, enter text in the wrong location, or struggle with non-standard and dynamic controls. That makes narrow tasks with explicit constraints easier to inspect than broad instructions such as "handle everything in this application."
For a development workflow, start with a read-only task, then test one controlled modification, and only after that consider a multi-step workflow. Keep the application visible while evaluating the feature and verify the result after each meaningful action. Avoid automatically allowing applications that contain sensitive information or perform high-impact operations. GitHub also warns that ambiguous instructions or unexpected screen content can lead to unintended actions affecting devices, data, connected accounts, or enterprise systems.Ā
The practical advantage is not that Copilot can replace every existing automation method. It is that there is now a documented route for bringing AI-assisted automation to software that was previously difficult to automate because its interface existed only on the screen. For developers, the sensible starting point is a small, reversible GUI task: enable computer use, verify permissions, give Copilot a read-only instruction, inspect the result, and only then allow a narrowly defined action.
Share this post
Written by



