Filtering In Unstructured Indexes
For documentation on unstructured indexes see here.
In unstructured indexes fields with numbers or booleans are automatically filterable. Fields with text however are only filterable if they are under the filterStringMaxLength
limit.
Example unstructured index with filtering
The following example shows how to create an unstructured index with filtering.
import marqo
settings = {
"filterStringMaxLength": 25,
}
mq = marqo.Client(url="http://localhost:8882")
mq.create_index("my-filt-unstructured-index", settings_dict=settings)
In this example we have set the filterStringMaxLength
to 25. This means that any text field with a length of 25 characters or less will be filterable. Any text field with a length greater than 25 characters will not be filterable.
Example with Add Documents and Search Usage
documents = [
{
"_id": "1",
"text": "The lazy brown fox jumps over the quick dog.",
"category": "story",
},
{
"_id": "2",
"text": "Marco Polo was an Italian merchant, explorer, and writer.",
"category": "this_category_is_longer_than_25_characters",
},
]
mq.index("my-filt-unstructured-index").add_documents(documents, tensor_fields=["text"])
# this works as story is less than 25 characters
results = mq.index("my-filt-unstructured-index").search(
q="dog", filter_string="category:story"
)
# this will not work (no results returned) as the category is longer than 25 characters
results = mq.index("my-filt-unstructured-index").search(
q="dog", filter_string="category:this_category_is_longer_than_25_characters"
)