Quick Start with Marqo

First, select your platform:

Marqo Cloud

Marqo Open Source

This guide is designed to get you working with Marqo Cloud quickly with just the basics. If you're looking for something for detailed and comprehensive, visit our Getting Started guide.

Full code:

If you have any questions or need help, visit our Community and ask in the get-help channel.

1. Install and Import

Sign Up to Marqo Cloud.
You will also need a Marqo API Key. In Marqo Cloud, navigate to API Keys. You can either create a new API key or use the default. Copy this key, you'll need it when we begin programming later. For a full walkthrough on how to find your API key, visit our article.

Now all that's left to do is install marqo:

pip install marqo

Now we can navigate to a Python script and begin importing Marqo:

import marqo

2. Create Marqo Client

Next, we create an instance of a client for interacting with the Marqo API. We'll specify the server URL, which in this case is running on Marqo Cloud at https://api.marqo.ai.

# Create a Marqo client
api_key = "your_api_key"
mq = marqo.Client("https://api.marqo.ai", api_key=api_key)

Note, api_key here is the API Key we obtained above. For a full walkthrough on obtaining your API Key, visit our article.

This step sets up the client to interact with the Marqo API, allowing us to perform various operations such as creating indexes and adding documents.

3. Housekeeping

Before we create a new index, it's good practice to delete any existing index with the same name to avoid conflicts. Here, we are deleting the "movies-index" if it already exists.

# Delete the index if it already exists
try:
    mq.index("movies-index").delete()
except:
    pass

This ensures that we start with a clean slate every time we run our script.

4. Creating a Marqo Index

Next, we create an index named "movies-index" using a specific machine learning model, hf/e5-base-v2. This model is designed to generate embeddings for various types of text inputs. It will be used for vectorizing the documents we add to the index.

# Create an index - Using this model: https://huggingface.co/intfloat/e5-base-v2
mq.create_index("movies-index", model="hf/e5-base-v2")

Creating an index is crucial as it prepares Marqo to store and manage the documents we'll be working with.

Your terminal should start populating with the following:

YYYY-MM-DD HH:MM:SS,mmm logger:'marqo' INFO Current index status: CREATING

This tells us that Marqo is creating our index.

The Marqo Cloud Indexes UI will also populate with this model:

Note that the status will turn to 'Ready' when the index is ready.

5. Adding Documents to the Marqo Index

Now, we add some movie descriptions to our index. These descriptions will be vectorized and stored in the index, making them searchable. We specify a 'Title' and 'Description' for each movie.

# Add documents (movie descriptions) to the index
mq.index("movies-index").add_documents(
    [
        {
            "Title": "Inception",  # Title of the movie
            "Description": "A mind-bending thriller about dream invasion and manipulation.",  # Movie description
        },
        {
            "Title": "Shrek",
            "Description": "An ogre's peaceful life is disrupted by a horde of fairy tale characters who need his help.",
        },
        {
            "Title": "Interstellar",
            "Description": "A team of explorers travel through a wormhole in space to ensure humanity's survival.",
        },
        {
            "Title": "The Martian",
            "Description": "An astronaut becomes stranded on Mars and must find a way to survive.",
        },
    ],
    # Specifies which fields of the documents should be used to generate vectors. In this case, 'Description'.
    tensor_fields=["Description"],
)

In this step, we specify that the "Description" field of each document should be used for vector search by including it in the tensor_fields parameter.

If we click on our index, we can see information such as the number of documents added.

As we see in our case, we've added 4 documents.

6. Search!

With our index populated with movie descriptions, we can now perform a search query. Let's search for a movie related to space exploration.

# Perform a search query on the index
results = mq.index("movies-index").search(
    # Our query
    q="Which movie is about space exploration?"
)

This query searches the descriptions in our index for content related to space exploration.

Finally, we print out the search results, including the title, description, and the relevance score for each movie that matches the query.

# Print the search results
for result in results["hits"]:
    print(
        f"Title: {result['Title']}, Description: {result['Description']}. Score: {result['_score']}"
    )

The relevance score (_score) indicates how well each document matches the search query.

Let’s look at the outputs:

Title: Interstellar, Description: A team of explorers travel through a wormhole in space to ensure humanity's survival.. Score: 0.8173517436600624
Title: The Martian, Description: An astronaut becomes stranded on Mars and must find a way to survive.. Score: 0.8081475581626953
Title: Inception, Description: A mind-bending thriller about dream invasion and manipulation.. Score: 0.7978701791216605
Title: Shrek, Description: An ogre's peaceful life is disrupted by a horde of fairy tale characters who need his help.. Score: 0.7619883916893311

Interstellar has the highest relevance score (0.817), indicating it is the most relevant to the query "Which movie is about space exploration?". The Martian follows closely with a score of 0.808, also highly relevant to the query. Inception and Shrek have lower scores (0.798 and 0.762, respectively), indicating they are less relevant to the space exploration theme. These scores help us understand how well each movie's description aligns with the search query, allowing us to identify the most pertinent results efficiently.

Calling an Existing Index in Marqo

Once you have created your index, you can call it again:

index = mq.index("movies-index")

Clean Up

When you no longer need your index, use the delete operation to delete it:

mq.index("movies-index").delete()

Support

Join our Slack community to ask questions and to chat with other community members about ideas!

Code

What's next?

This quick start tutorial showed you how to perform a simple text search with Marqo. If you're looking for something more comprehensive, visit our Getting Started guide.

If you're happy with the basics, why don't you try searching with different modalities:

This guide is designed to get you working with Marqo Open Source quickly with just the basics. If you're looking for something more detailed and comprehensive, visit our Getting Started guide.

Full code: Marqo Open Source Quick Start Code

If you have any questions or need help, visit our Community and ask in the get-help channel.

1. Installing

We first need to install Marqo. If you have followed our Installation Guide, you can skip this step.

Marqo requires Docker. To install Docker go to Docker Docs and install for your operating system (Mac, Windows, Linux).
Once Docker is installed, you can use it to run Marqo. First, open the Docker application and then head to your terminal and enter the following:
```
docker pull marqoai/marqo:latest
docker rm -f marqo
docker run --name marqo -it -p 8882:8882 marqoai/marqo:latest
```
Once everything is complete, your terminal should look like this:

While Docker is running, we can use Marqo as we would any other Python library. First, pip install it:

pip install marqo

2. Create Marqo Client

Next, we need to create a Marqo client that will communicate with the Marqo server. We'll specify the server URL, which in this case is running locally on http://localhost:8882. Head to a Python script and input the following:

import marqo

# Create a Marqo client
mq = marqo.Client(url="http://localhost:8882")

This step sets up the client to interact with the Marqo API, allowing us to perform various operations such as creating indexes and adding documents.

Before we create a new index, it's good practice to delete any existing index with the same name to avoid conflicts. Here, we are deleting the "movies-index" if it already exists.

# Delete the index if it already exists
try:
    mq.index("movies-index").delete()
except:
    pass

This ensures that we start with a clean slate every time we run our script.

3. Creating a Marqo Index

Next, we create an index named "movies-index" using a specific machine learning model, hf/e5-base-v2. This model is designed to generate embeddings for various types of text inputs. It will be used for vectorizing the documents we add to the index.

# Create an index - Using this model: https://huggingface.co/intfloat/e5-base-v2
mq.create_index("movies-index", model="hf/e5-base-v2")

Creating an index is crucial as it prepares Marqo to store and manage the documents we'll be working with.

4. Adding Documents to the Marqo Index

Now, we add some movie descriptions to our index. These descriptions will be vectorized and stored in the index, making them searchable. We specify a 'Title' and 'Description' for each movie.

# Add documents (movie descriptions) to the index
mq.index("movies-index").add_documents(
    [
        {
            "Title": "Inception",  # Title of the movie
            "Description": "A mind-bending thriller about dream invasion and manipulation.",  # Movie description
        },
        {
            "Title": "Shrek",
            "Description": "An ogre's peaceful life is disrupted by a horde of fairy tale characters who need his help.",
        },
        {
            "Title": "Interstellar",
            "Description": "A team of explorers travel through a wormhole in space to ensure humanity's survival.",
        },
        {
            "Title": "The Martian",
            "Description": "An astronaut becomes stranded on Mars and must find a way to survive.",
        },
    ],
    # Specifies which fields of the documents should be used to generate vectors. In this case, 'Description'.
    tensor_fields=["Description"],
)

In this step, we specify that the "Description" field of each document should be used for vector search by including it in the tensor_fields parameter.

5. Search!

With our index populated with movie descriptions, we can now perform a search query. Let's search for a movie related to space exploration.

# Perform a search query on the index
results = mq.index("movies-index").search(
    # Our query
    q="Which movie is about space exploration?"
)

This query searches the descriptions in our index for content related to space exploration.

Finally, we print out the search results, including the title, description, and the relevance score for each movie that matches the query.

# Print the search results
for result in results["hits"]:
    print(
        f"Title: {result['Title']}, Description: {result['Description']}. Score: {result['_score']}"
    )

The relevance score (_score) indicates how well each document matches the search query.

Let’s look at the outputs:

Title: Interstellar, Description: A team of explorers travel through a wormhole in space to ensure humanity's survival.. Score: 0.8173517436600624
Title: The Martian, Description: An astronaut becomes stranded on Mars and must find a way to survive.. Score: 0.8081475581626953
Title: Inception, Description: A mind-bending thriller about dream invasion and manipulation.. Score: 0.7978701791216605
Title: Shrek, Description: An ogre's peaceful life is disrupted by a horde of fairy tale characters who need his help.. Score: 0.7619883916893311

Interstellar has the highest relevance score (0.817), indicating it is the most relevant to the query "Which movie is about space exploration?". The Martian follows closely with a score of 0.808, also highly relevant to the query. Inception and Shrek have lower scores (0.798 and 0.762, respectively), indicating they are less relevant to the space exploration theme. These scores help us understand how well each movie's description aligns with the search query, allowing us to identify the most pertinent results efficiently.

Calling an Existing Index in Marqo

Once you have created your index, you can call it again using:

index = mq.index("movies-index")

Clean Up

When you no longer need your index, use the delete operation to delete it:

mq.index("movies-index").delete()

Support

Join our Slack community to ask questions and to chat with other community members about ideas!

Code

For the code used in this Quick Start, see here.

What's next?

This quick start tutorial showed you how to perform a simple text search with Marqo. If you're looking for something more comprehensive, visit our Getting Started guide.

If you're happy with the basics, try searching with different modalities: