Skip to main content

Image Detection API Documentation

Overview​

The Image detection API identifies objects in images and returns bounding boxes with confidence scores and labels.

Guide Prerequisites​

  • Marqo Cloud Account: Sign up at cloud.marqo.ai. (if you don't have an account, please reach out!)
  • Python 3.7+ installed on your machine
  • Image Detection API Key given to you by your Marqo representative. Please reach out to the team if you don't have one.

Base URL​

https://image-detection.marqo-ep.ai

Authentication​

All API requests require authentication using an API key passed in the request header:

x-api-key: <your-api-key>
note

Image Detection runs as a separate service on its own host, with its own credential. The x-api-key above is not your Ecom API key and not an index ID, and the Ecom API credentials described in Responses & Errors are not accepted here. Ask your Marqo representative for an Image Detection key.

Endpoints​

Detect Objects​

Detects objects in an image and returns bounding boxes with confidence scores and labels.

Endpoint: POST /detect

Headers:

  • Content-Type: application/json
  • x-api-key: <your-api-key>

Request Body:

{
  "image": "<image_url_or_base64>"
}

Parameters:

  • image (string, required): Either a URL to an image or a base64-encoded image string

Supported Image Formats:

  • JPEG
  • WebP
  • PNG

Example Request:

curl -X POST 'https://image-detection.marqo-ep.ai/detect' \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: your-api-key-here' \
  --data '{
    "image": "https://example.com/image.jpg"
  }'

Example Response:

{
  "items": [
    {
      "bounding_box": [
        [0.1, 0.15],
        [0.3, 0.4]
      ],
      "score": 0.95,
      "labels": ["bag", "handbag"]
    },
    {
      "bounding_box": [
        [0.2, 0.1],
        [0.4, 0.3]
      ],
      "score": 0.87,
      "labels": ["top", "shirt"]
    }
  ],
  "original_image": {
    "width": 800,
    "height": 600
  }
}

Response Fields:

  • items (array): Array of detected objects
    • bounding_box (array): Coordinates of the bounding box using top-left corner as origin (0,0)
      • First array: [x1, y1] - Top-left corner coordinates
      • Second array: [x2, y2] - Bottom-right corner coordinates
    • score (float): Confidence score between 0 and 1
    • labels (array): Array of category labels for the detected object
  • original_image (object): Original image dimensions
    • width (integer): Image width in pixels
    • height (integer): Image height in pixels

Error Responses​

400 Bad Request​

{
  "error": "Missing 'image' field in request"
}

401 Unauthorized​

{
  "error": "Invalid or missing API key"
}

405 Method Not Allowed​

{
  "error": "Invalid request method"
}

After detecting objects in an image, you can:

  1. Extract the bounding box coordinates from the response
  2. Crop the original image using the bounding box coordinates
  3. Convert the cropped image to base64
  4. Submit the cropped image as the query to one of the two image search endpoints on the Ecom API:
    • Reverse Image Search ranks candidates with an AI visual-similarity assessment. Use it for photographs taken in the wild, which is what a cropped detection usually is.
    • Search by Image on /search runs a plain image query through the normal search pipeline. Use it when you want the same endpoint, response shape and pagination as a text search.

Both endpoints are part of the Ecom API and use the Ecom API credentials, not the Image Detection key above.