AI is moving out of the cloud and onto local machines, promising faster, more private decision-making for critical tasks. This new capability is being driven by models like D3, a family of open-weight AI designed not to generate text, but to return probabilities for decisions.
D3 is a series of open-weight decision models that, instead of generating text, return probabilities for each answer. It can process tasks like classifying support tickets, scoring articles, or checking video clips in a single request to a model running on a local machine, with our tests showing responses in as little as 87 milliseconds. The suite consists of six models, ranging from 600 million to 27 billion parameters, allowing small models to be experimented with on affordable hardware and larger models for more demanding tasks. D3 Edge is so lightweight it can run on compatible smartphones.
What is a D3 Decision Model?
D3 is an open-weight AI model series designed for decision-making, not for text generation. For example, instead of asking a large language model (LLM) to explain if a transaction is suspicious, D3 can directly return the probability that it needs review. It supports three decision types: Choice, for selecting from predefined options; Noul, for returning yes/no probabilities; and Score, for evaluating on a sequential scale. The D3 family is available in six sizes, from 600 million to 27 billion parameters, suitable for environments ranging from edge devices to high-end GPUs.
Running D3 Locally with vLLM
vLLM’s new Structured Decisions support is used to run D3 locally without installing the Semantic Router itself. vLLM serves the model through a fast and intuitive “System 1” API named System One, which answers typed questions in a single pass and returns probabilities instead of generated text. A tutorial demonstrates this process on Ubuntu Linux with Python 3.12 and an NVIDIA RTX 3090 GPU. After installing dependencies like `uv` and the nightly build of vLLM, the D3 Mini model is started with a GPU memory setting suitable for local testing. Once loaded, vLLM exposes a local System One API at `http://localhost:8000/v1/systemone`.
Building Projects with AI Tools
A TypeSafe Python SDK is used to communicate with the locally running model to explore classification, probability estimation, and scoring methods. In one example, a D3 model returns a 99.9% probability that a suspicious transaction should be flagged for fraud review, a process that took about 87 milliseconds. The tutorial guides users through building three practical projects: an AI blog quality analyzer using TypeSafe Python SDK and Olostep to classify and evaluate real articles; an image understanding tool to analyze product images for category identification and quality assessment; and a video understanding tool to detect motion and visual changes in short clips. Our tests showed response times between 87 milliseconds and 1.11 second.
