Filtering In Unstructured Indexes

For documentation on unstructured indexes see here.

In unstructured indexes fields with numbers or booleans are automatically filterable. Fields with text however are only filterable if they are under the filterStringMaxLength limit.

Example unstructured index with filtering

The following example shows how to create an unstructured index with filtering.

import marqo

settings = {
    "filterStringMaxLength": 25,
}

mq = marqo.Client(url="http://localhost:8882")

mq.create_index("my-filt-unstructured-index", settings_dict=settings)

In this example we have set the filterStringMaxLength to 25. This means that any text field with a length of 25 characters or less will be filterable. Any text field with a length greater than 25 characters will not be filterable.

Example with Add Documents and Search Usage

documents = [
    {
        "_id": "1",
        "text": "The lazy brown fox jumps over the quick dog.",
        "category": "story",
    },
    {
        "_id": "2",
        "text": "Marco Polo was an Italian merchant, explorer, and writer.",
        "category": "this_category_is_longer_than_25_characters",
    },
]

mq.index("my-filt-unstructured-index").add_documents(documents, tensor_fields=["text"])

# this works as story is less than 25 characters
results = mq.index("my-filt-unstructured-index").search(
    q="dog", filter_string="category:story"
)

# this will not work (no results returned) as the category is longer than 25 characters
results = mq.index("my-filt-unstructured-index").search(
    q="dog", filter_string="category:this_category_is_longer_than_25_characters"
)