Text Only Indexes
For documentation on structured indexes see here.
In this section we will go over how to create a simple structured index with a text field to vectorise. This is the simplest form of a structured index. We will talk through each part of the settings object and what it does.
The bare minimum settings object for a structured index with a text field that supports tensor and lexical search is as follows:
import marqo
settings = {
"type": "structured",
"allFields": [
{"name": "text_field", "type": "text", "features": ["lexical_search"]},
],
"tensorFields": ["text_field"],
}
mq = marqo.Client(url="http://localhost:8882")
mq.create_index("my-simple-structured-index", settings_dict=settings)
Index Settings: type
The Index Settings: type
field is required and must be set to structured
for a structured index.
Index Settings: allFields
The allFields
field defines the schema for the index. Each field in the schema must have a name
and a type
, they can optionally have a features
field as well.
The name
defines the name of the field in the document. The type
defines the datatype of the field. The features
field is an array of features that can be applied to the field. In this case we are using the lexical_search
feature which allows us to search with the LEXICAL
search method.
Index Settings: tensorFields
The tensorFields
field is an array of fields that will be vectorised. In this case we are vectorising the text_field
field. These fields are available for the TENSOR
search method.
Example Add Documents Usage
Because we are using a structured index we don't need to specify the tensor fields. The schema will take care of this for us.
documents = [
{
"_id": "1",
"text_field": "New York",
},
{
"_id": "2",
"text_field": "Los Angeles",
},
]
mq.index("my-simple-structured-index").add_documents(documents)
Example search usage
The schema above will allow us to search the text_field
field using the LEXICAL
search method and the TENSOR
search method.
Tensor Search (search_method="TENSOR"
is the default):
results = mq.index("my-simple-structured-index").search(q="New York")
Lexical Search:
results = mq.index("my-simple-structured-index").search(
q="New York", search_method="LEXICAL"
)