Amazon S3 Vectors now supports metadata pre-filtering, a capability that evaluates filters before similarity searches to deliver higher recall on targeted queries.
Resolving Vector Scope Before the Similarity Search
Most applications never search an entire vector index. Instead, they query a subset belonging to a specific user, account, or document category. Semantic search, retrieval-augmented generation (RAG), and autonomous agent systems all rely on filtered queries to scope similarity searches. Before this update, Amazon S3 Vectors operated primarily in an index mode designated as CLASSIC. In that mode, the underlying infrastructure performed the vector similarity search and the metadata filter evaluation in tandem, validating candidate vectors against filters concurrently during the search sweep.
Under the new update, indexes configured to an ENHANCED index mode evaluate the metadata filter first. By resolving constraints prior to executing the vector math, the system searches only the subset of vectors that match the criteria. For instance, in a support knowledge base containing 8 million tickets where a single customer accounts for 400 records, resolving the customer ID first ensures the similarity search scans all 400 relevant vectors. Previously, candidate pools drawn from the full 8 million records often resulted in fewer matching tickets surfacing in the final output.
Enhanced Filter Mechanisms for Hierarchical Data
The update introduces prefix matching capabilities through the $startsWith operator, designed specifically for hierarchical keys, paths, and URLs. This operator joins existing filter mechanisms, including equality checks, numeric ranges, set memberships, existence checks, and boolean logic handled via $and and $or operators.
Each vector can carry up to 2 KB of application-defined metadata. A single query supports up to 100 individual filter constraints. Developers can implement these filters without declaring an upfront schema, as every metadata field is filterable by default.
Consider how legal and professional services handle document management. A platform searching an e-discovery archive can scope documents to a single client and use $startsWith to narrow results further by matter number or folder path. Similar structural scoping applies across financial services, where research platforms filter analyst notes by issuer and publication date, and in media streaming architectures, where catalogs filter assets by content rating and regional licensing windows before running semantic matching.
Upgrading Existing Indexes to Enhanced Mode
Transitioning an existing vector index to utilize pre-filtering requires updating its operational mode. Developers can execute an index mode modification without re-ingesting existing data or altering query syntax.
aws s3vectors update-index-mode
--vector-bucket-name my-vector-bucket
--index-name product-catalog
--index-mode ENHANCED