Loading Sentence Transformers from Hugging Face

Loading other Models

Other models can be loaded from Hugging Face into Marqo, there are just a few things to check first. For this example we will use BAAI/bge-small-en-v1.5.

Checking API Compatibility

Once you have found a model you would like to try, check that it is compatible with the sentence_transformers API. You can do this by checking the model's page on Hugging Face and looking for the "Use in libraries" or "Use in sentence-transformers" button above the downloads graph. If sentence-transformers appears then the model should work in Marqo.

Checking Model Details

To load a model into Marqo you will need to know its embedding dimension and max tokens. This can be found by navigating to the "Files and versions" tab and opening the config.json file.

The embedding dimension is the value of hidden_size and the max tokens is the value of max_position_embeddings. Below is an example of the config for BAAI/bge-small-en-v1.5:

{
  "_name_or_path": "/root/.cache/torch/sentence_transformers/BAAI_bge-small-en/",
  "architectures": [
    "BertModel"
  ],
  "attention_probs_dropout_prob": 0.1,
  "classifier_dropout": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 384,
  "id2label": {
    "0": "LABEL_0"
  },
  "initializer_range": 0.02,
  "intermediate_size": 1536,
  "label2id": {
    "LABEL_0": 0
  },
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "torch_dtype": "float32",
  "transformers_version": "4.30.0",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 30522
}

Loading the Model

To use the model in Marqo you can use the modelProperties object in the index settings with the model name from Hugging Face and the other details we extracted from the config.json. e.g.

import marqo

settings = {
    "modelProperties": {
        "name": "BAAI/bge-small-en-v1.5",
        "type": "hf",
        "dimensions": 384,
        "tokens": 512,
    },
    "model": "bge-small-en-v1.5",
}

mq = marqo.Client()

mq.create_index("my-first-index", settings_dict=settings)

# trigger the model download
mq.index("my-first-index").search("query: apple")