Skip to main content

Filtering in Marqo

You can use Marqo's filter string DSL to refine search results. Filter strings can be used with all search methods.

Filters have several use cases, for example, restricting the results a specific user has access to or creating faceted search interfaces.

Filters use a syntax to parse and split the provided filter string based on operators, such as AND or NOT. Marqo then analyzes each split text independently before returning matching documents.

Example​

Pass the filter string in the filter parameter of a search request. In the following example, Marqo's filter string analyzer splits the filter string into two components, brand:(New Balance) and price:[0 TO 150].

curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/search \
-H "x-marqo-index-id: {index_id}" \
-H "Content-Type: application/json" \
-d '{
  "q": "running shoes",
  "filter": "brand:(New Balance) OR price:[0 TO 150]"
}'

The same filter parameter is accepted by collections, reverse image search and the other search endpoints. Any filter configured on the index is combined with the filter you send using AND.

Marqo's filter string DSL is based on Lucene but with some differences.

  • all term filters must be connected to a field, e.g., city:(New York)

  • efficient range filters are supported for numeric types without manually specifying the type

  • fuzzy/approximate searches are not supported

Let's dig into some of the specifics:

Fields​

Marqo's search terms must be fielded. You can search any field by typing the field name followed by a colon ":" and then the term you are looking for.

As an example, let's assume a Marqo index contains two fields, title and text and text is the default field. If you want to find the document entitled "The Right Way" which contains the text "go", you can enter:

title:(The Right Way) AND text:go

Since text is the default field, the field indicator is not required.

Note: The field is only valid for the term that it directly precedes, so the filter string

title:Do it right

Will only find "Do" in the title field. It will find "it" and "right" in the default field (in this case the text field).

Range filters​

Marqo supports efficient execution of range filters on numeric types.

Numbers from 0..100:

some_numeric:[0 TO 100]

Greater than or equal to 0:

some_numeric:[0 TO *]

IN filters​

Marqo supports the IN operator for restricting a field to a list of values. The value list must be enclosed in parentheses and comma-separated. Values with spaces must be enclosed in parentheses.

IN works on text, numeric and array fields. The _id field can also be filtered on with IN.

Text field example:

text_field IN (apple, banana, (wild cherry))

Integer field example:

int_field IN (1, 2, 3)

CONTAINS filters​

Marqo supports the CONTAINS operator for matching documents where a text field contains a given word or phrase. Unlike an equality filter, which matches the field's exact value, CONTAINS performs token-level matching against the field's content.

title CONTAINS hello

This matches any document whose title field contains the word "hello" anywhere in the text, for example "Hello World" or "hello again". Matching is case-insensitive, and the CONTAINS keyword itself is also case-insensitive (contains and Contains work too).

Matching is performed on whole tokens (words), not substrings. For example, title CONTAINS ello will not match a document with the title "Hello World".

To match a multi-word phrase, enclose the value in parentheses:

description CONTAINS (simple greeting)

This matches documents where the words "simple greeting" appear as a phrase in the description field.

CONTAINS terms can be combined freely with Boolean operators, grouping, and other filter types:

title CONTAINS hello AND score:[1 TO 10]
NOT (title CONTAINS hello) AND title CONTAINS world

Boolean filters​

Marqo supports execution of boolean filters, if you reference true or false within the filter then it will be treated as a boolean.

some_bool:true

Boolean Operators​

Boolean operators allow terms to be combined through logic operators. Marqo supports AND, "+", OR, NOT and "-" as Boolean operators(Note: Boolean operators must be ALL CAPS).

The OR operator is the default conjunction operator. This means that if there is no Boolean operator between two terms, the OR operator is used. The OR operator links two terms and finds a matching document if either of the terms exist in a document. This is equivalent to a union using sets. The symbol || can be used in place of the word OR.

food:(ice cream) OR type:confectionary

The AND operator matches documents where both terms exist anywhere in the text of a single document. This is equivalent to an intersection using sets. The symbol && can be used in place of the word AND.

To search for documents that exactly match "ice cream" and "confectionary" use the filter string:

food:(ice cream) AND type:confectionary

The NOT operator excludes documents that contain the term after NOT. This is equivalent to a difference using sets. The symbol ! can be used in place of the word NOT.

To search for documents that match a type of "confectionary" but not "ice cream" use the filter string:

type:confectionary AND NOT food:(ice cream)

Grouping​

Marqo filtering supports using parentheses to group clauses. This can be very useful if you want to control the boolean logic for a filter string.

To search for either type is "confectionary" or food is "ice cream" and sweetness is 10 use the filter string:

(type:confectionary OR food:(ice cream)) AND sweetness:10

Escaping Special Characters​

Marqo supports escaping special characters that are part of the filter string syntax. The current list special characters are

+ - && || ! ( ) { } [ ] ^ " ~ * ? : \

To escape these character use backslash (\) before the character. For example to filter for (1+1):2 in the field myField use the filter string:

myField:\(1\+1\)\:2

Note that inside a JSON string the backslashes themselves must be escaped, so the request body carries each backslash doubled:

curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/search \
-H "x-marqo-index-id: {index_id}" \
-H "Content-Type: application/json" \
-d '{
  "q": "what colour are plants?",
  "filter": "myField:\\(1\\+1\\)\\:2"
}'

You also need to escape characters such as the special characters and spaces in fieldnames:

my\ field:hello

Filtering with array fields​

Marqo supports filtering over array fields. This can be useful for use cases such as filtering over product collections or tags. A term matches when any element of the array equals the value.

productCollections:sale
productCollections IN (sale, (new arrivals))