The rise of AI-powered coding assistants like GitHub Copilot has dramatically changed the software development landscape, offering real-time code suggestions and accelerating workflows. Though, concerns around data privacy and reliance on cloud services have prompted developers to explore alternatives. It’s now possible to achieve similar code completion capabilities directly within Visual Studio Code, leveraging locally hosted Large Language Models (LLMs) – and you don’t need to subscribe to Copilot to do it.
For developers prioritizing control and privacy, running an LLM locally offers a compelling solution. This approach keeps your code and data on your machine, eliminating the need to transmit information to external servers. Recent advancements in the VS Code AI Toolkit extension have made this process significantly more accessible, removing the need for separate software installations like Ollama or LM Studio. Setting up local LLMs in VS Code is becoming increasingly streamlined, opening up AI-assisted coding to a wider audience.
Previously, utilizing local LLMs within VS Code often required configuring tools like Ollama or LM Studio and establishing a connection through a local port. Now, the VS Code AI Toolkit extension provides a direct pathway to download and run models without these intermediary steps. The extension, designed for experimenting with both cloud-based and local models, simplifies the process considerably. Installation is straightforward via the extensions tab within VS Code, after which users can access a model catalog.
Within the AI Toolkit extension, the model catalog allows filtering by various criteria, including whether the model is hosted locally or in the cloud and if local, whether it’s designed to run on a CPU, GPU, or Neural Processing Unit (NPU). Users with Copilot+ PCs, such as the Asus Proart P16, can accept advantage of their NPU for efficient local model execution. However, the toolkit supports GPU and CPU-based models as well, ensuring compatibility across a range of hardware configurations. Once a model – like Phi-4 mini reasoning or Qwen Coder – is selected and added, it’s downloaded and can be activated directly within VS Code.
To begin using the downloaded model, simply navigate to the “Models” section under “MY RESOURCES” within the AI Toolkit and double-click the desired model to load it. With the model loaded, VS Code’s Copilot feature can then be activated, providing AI-powered code completion and suggestions driven by the local LLM. This setup allows developers to enjoy the benefits of AI assistance while maintaining complete control over their code and data.
GitHub Copilot itself still requires a subscription to apply, as outlined in the official VS Code documentation. However, the AI Toolkit extension provides a pathway to leverage the Copilot interface with locally hosted models, offering an alternative for those seeking a privacy-focused solution. The free version of Copilot does have telemetry enabled by default, which can be disabled via VS Code settings.
The ability to run LLMs locally within VS Code represents a significant step towards democratizing AI-assisted coding. By removing the reliance on cloud services and simplifying the setup process, developers can now harness the power of AI while maintaining greater control over their data and workflows. Tools like Ollama continue to provide alternative methods for deploying local LLMs, as detailed in guides on building custom local Copilot extensions, but the VS Code AI Toolkit offers a more integrated and user-friendly experience.
As LLMs continue to evolve and hardware capabilities improve, we can expect even more seamless integration of local AI models into development environments. The trend towards on-device AI processing is likely to accelerate, empowering developers with greater flexibility, privacy, and control over their coding workflows. The future of AI-assisted coding is increasingly localized, offering a compelling alternative to cloud-based solutions.
What are your thoughts on running LLMs locally? Share your experiences and insights in the comments below.
