Pinecone : Metadata filtering
Metadata filtering in the context of vector databases is a technique used to refine search results or queries based on additional information (metadata) associated with the vectors stored in the database. In a vector database, each entry typically consists of a high-dimensional vector representing a data item (like an image, text snippet, or audio clip) along with its associated metadata.
Let’s try to understand this with an example.
We have here vectorized versions of blogs with there categories and published date. We will be pushing this data in Pinecone. We can then perform query search operation using the metadata for filters.
Here , we can see our vectors inserted in Pinecone index named “pinecone-metadata-filtering.”
Let’s now perform a normal query without metadata filtering.
We have got the closest top 3 vectors to the query vector as per the set vector similarity metric. It can be cosine similarity , euclidean distance or dot product similarity for Pinecone.
Now , let’s try querying it using metadata. We will set our filter condition and then pass it as parameter in the filter condition.
Here we have set the filter condition as category equal to marketing and published greater 2015 and in our results we have see the conditions are being met.This was a quick busy person’s guide to Metadata filtering in general though here I have used Pinecone for demo.
Reference : https://docs.pinecone.io/docs/metadata-filtering
