Reverse Image Search

Reverse image search finds the products in your catalog that look most like a query image. It uses AI to assess visual similarity and order the results so the closest matches come first, which makes it well suited to "find this exact product" experiences. Because of the additional AI processing, it is a bit slower than a plain Search by Image request, but is able to get better results for "in-the-wild" images that users upload.

note

Reverse image search must be enabled for your index before it can be used. If it is not enabled, the endpoint returns 404. Contact your Marqo account manager to enable this feature.

Prerequisites

• A Marqo Cloud account (sign up here)
• Your Marqo index ID (used for the x-marqo-index-id header)
• Reverse image search enabled for your index — contact your Marqo account manager to enable this feature

Endpoint

Requests are made via a POST to the reverse-image-search endpoint. The query image is provided in q, either as a URL to an image or as a data URL containing a base64-encoded image.

cURL

# With a URL to an image
curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/reverse-image-search \
-H "x-marqo-index-id: {index_id}" \
-H "Content-Type: application/json" \
-d '{
  "q": "https://example.com/image.jpg",
  "limit": 10
}'

# With a base64 data URL
curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/reverse-image-search \
-H "x-marqo-index-id: {index_id}" \
-H "Content-Type: application/json" \
-d '{
  "q": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD//gAMQX..."
}'

JavaScript

fetch(
  "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/reverse-image-search",
  {
    method: "POST",
    headers: {
      "x-marqo-index-id": "{index_id}",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      q: "https://example.com/image.jpg",
      limit: 10,
    }),
  }
);

Note: Replace {index_name} with your actual index name and {index_id} with the ID provided for your ecommerce deployment.

Response

Hits are returned best-first. Each hit has the same shape as a regular /search hit.

Consume the hits in the order the array gives them. Normally that order is the AI assessment's visual-similarity ranking, but when the assessment cannot complete the endpoint still returns 200 with the candidates in retrieval order. See Behaviour notes for when that happens.

{
  "hits": [
    {
      "productTitle": "Premium Wireless Bluetooth Headphones",
      "variantTitle": "Premium Wireless Bluetooth Headphones - Black",
      "price": 199.99,
      "variantImageUrl": "https://cdn.example.com/headphones-black.jpg",
      "collections": ["electronics", "audio"],
      "_id": "headphones-black-001"
    }
  ],
  "limit": 10,
  "processingTimeMs": 1830
}

Response fields

Field	Type	Description
hits	Array	Matching products, best-first. Use the array order as given
limit	Integer	The number of results requested
processingTimeMs	Integer	End-to-end time the request took on the server, in milliseconds. Unlike processingTimeMs on /search, this includes the candidate searches and the AI assessment, so expect values in the seconds range.

Behaviour notes

• Only products with a variantImageUrl can be assessed for visual similarity. Candidates without one are not returned, even if they match the filter.
• The index's configured search settings apply to the candidate searches, including any configured filter and collapse fields. A request filter is combined with the configured filter, so both must match.
• Do not sort or threshold on _score. When a hit carries one, it is the score of the internal candidate search that surfaced that product, not a measure of visual similarity to the query image. It does not follow the returned order. The AI assessment's own scores are not returned.
• The endpoint degrades rather than failing. The query image is described several ways and each description drives its own candidate search. If one of those descriptions cannot be produced in time, that line of enquiry is dropped and the rest still run. If none of them contributes candidates, or if the AI assessment fails or runs out of time, the response is still 200 but the hits come back in retrieval order rather than assessed order. Nothing in the response distinguishes the two, so treat the ordering as best-effort.
• When it does fail. A 500 is returned only if every candidate search fails, or if an overall time budget is configured for your index and the request exceeds it.
• Redirects, synonyms, merchandising rules and personalization are not applied on this endpoint. A userId in the request has no effect and no x-marqo-results-personalized header is returned.
• Responses do not carry the x-marqo-cache-hit header. See Responses & Errors for the headers other endpoints return.

Parameters

Parameter	Type	Default	Description
q	String	required	The query image, as an image URL or a base64 data URL. Text queries are not accepted.
limit	Integer	10	Maximum number of products to return. Must be an integer between 1 and 10; other values return 400.
filter	String	null	Filter string using Marqo's query DSL. Only products matching the filter are returned.
attributesToRetrieve	Array of strings	Default fields	Specific product fields to return on each hit. If not specified, returns the default core product fields.

Image requirements

When q is a URL, the path must end in one of these image extensions: .jpg, .jpeg, .png, .webp, .tiff, .avif. When q is a data URL, any image/* MIME type is accepted.

Filtering

You can narrow the results with a filter string. Only products matching the filter are returned.

cURL

curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/reverse-image-search \
-H "x-marqo-index-id: {index_id}" \
-H "Content-Type: application/json" \
-d '{
  "q": "https://example.com/sneaker.jpg",
  "filter": "brand:Nike AND price:[* TO 200]"
}'

JavaScript

fetch(
  "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/reverse-image-search",
  {
    method: "POST",
    headers: {
      "x-marqo-index-id": "{index_id}",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      q: "https://example.com/sneaker.jpg",
      filter: "brand:Nike AND price:[* TO 200]",
    }),
  }
);

For the full filter syntax, see the Filter DSL Guide.

Error responses

Status	Body	Condition
400	{ "error": "q must be an image URL or base64 data URI" }	q is missing or is not a valid image URL or data URL
400	{ "error": "filter must be a string" }	filter was provided but is not a string
400	{ "error": "attributesToRetrieve must be an array of strings" }	attributesToRetrieve was provided but is not an array of strings
400	{ "error": "limit must be an integer between 1 and 10" }	limit was provided but is not an integer in the range 1 to 10
401	{ "error": "Unauthorized" }	Missing or invalid credentials
404	{ "error": "reverse image search is not enabled for this index" }	Reverse image search is not enabled for the index
429	See below	The index has exceeded 200 reverse image search requests in the last minute

Errors shared by every endpoint (404 for an unknown index, 500, 502) are listed on Responses & Errors.

Rate limit

Reverse image search is limited to 200 requests per minute per index. Requests over the limit are rejected before any processing with 429 and a Retry-After: 60 header. The limit is counted independently in each data centre that serves your traffic, so treat 200 as the figure to pace against rather than an exact ceiling. Unlike other errors, the 429 body does not use the error key:

{
  "type": "RateLimitExceeded",
  "userMessage": "Rate limit exceeded for reverse image search; retry after 60 seconds",
  "status": 429,
  "extra": { "window_seconds": 60 }
}

Wait window_seconds seconds before retrying. Browser scripts cannot read the Retry-After header on cross-origin responses, so use the body value there.