Monitor Jobs

This guide shows you how to monitor asynchronous jobs in your Marqo ecommerce search index. Learn about job statuses, how to retrieve job information, and best practices for tracking document operations.

Prerequisites

A Marqo Cloud account (sign up here)
Your Marqo API key (find your API key guide)
An existing ecommerce index (create an index guide)

Understanding Asynchronous Jobs

When you perform document operations (adding, updating, or deleting products), these operations are asynchronous. This means:

Immediate Response: Your request is validated and accepted immediately, and you receive a job ID (HTTP status code 202 Accepted)
Background Processing: The actual operation happens in the background
Status Tracking: You can monitor progress using the job ID
Error Handling: Per-document success or failure details are recorded on the job once processing completes. They are not part of the 202 response.

Both endpoints below authenticate with Authorization: Bearer {api_key}.

Job Operations

List Jobs

Retrieve recent jobs for your index (by default spanning the last 24 hours):

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs \
-H "Authorization: Bearer {api_key}"

Get Specific Job

Retrieve details for a specific job using its job ID:

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs/{job_id} \
-H "Authorization: Bearer {api_key}"

An unknown job ID returns 404 with {"error": "Job not found"}.

If the index name itself is unknown, belongs to another account, or is not ready to serve requests yet, both job endpoints return 404 with {"error": "Marqo settings not found for shop {system_account_id}-{index_name}"} instead.

:::warning Do not send x-marqo-internal Writes made with the x-marqo-internal: true header create jobs that are hidden from /jobs unless the same header is sent on the listing request. If you copy this header from an internal example, your own jobs will appear to be missing. :::

Filter Jobs by Status

Get Pending Jobs

Jobs that are queued but not yet started:

curl "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=PENDING" \
-H "Authorization: Bearer {api_key}"

Get In Progress Jobs

Jobs that are currently running:

curl "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=IN_PROGRESS" \
-H "Authorization: Bearer {api_key}"

Get Jobs with Rejected Documents

Jobs in which some documents were rejected for a reason on your side (for example an image that could not be downloaded):

curl "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=ERROR" \
-H "Authorization: Bearer {api_key}"

Get Failed Jobs

Jobs in which some documents failed for a server-side reason:

curl "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=FAILED" \
-H "Authorization: Bearer {api_key}"

Filter by Time Range

You can combine filters with a time window to focus on a specific period (useful for alerting and incident reviews). The /jobs endpoint accepts the following query parameters:

Parameter	Type	Default	Description
`status`	String (optional)	all statuses	Filter to `PENDING`, `IN_PROGRESS`, `COMPLETED`, `ERROR`, `FAILED`, `CONFLICT`, or `CANCELLED`. Any other value is rejected with `422`.
`timeFrom`	ISO 8601 string (optional)	24 hours ago	Start of the results window (e.g. `2025-11-06T01:10:12+00:00`). Must be within the last 30 days and not in the future.
`timeTo`	ISO 8601 string (optional)	now	End of the results window. Must not be in the future.
`limit`	Integer (optional)	`10`	Maximum number of jobs to include, most recent first. Maximum `1000`; a higher value is rejected with `422`. `0` means no limit, except when `status` is also set, where it is capped at 100 jobs. There is no pagination: `total` tells you how many jobs matched the window, but jobs beyond `limit` are not returned, and `jobsByStatus` and `docsByStatus` are aggregated over the returned jobs only. To see older jobs, narrow the window with `timeFrom` and `timeTo`.

Example: retrieve up to 100 failed jobs from a one-week period (use timestamps within the last 30 days):

curl "https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs?status=FAILED&timeFrom={iso_time_from}&timeTo={iso_time_to}&limit=100" \
-H "Authorization: Bearer {api_key}"

An out-of-range limit or an unrecognised status is rejected with 422 and a list of validation issues. An invalid window is rejected with 400 and an {"error": ...} body:

`error`	When
`"time_from must be within the last 30 days"`	`timeFrom` is older than 30 days
`"time_from must be in the past"`	`timeFrom` is in the future
`"time_to must not be in the future"`	`timeTo` is in the future
`"time_from must be before time_to"`	`timeFrom` is later than `timeTo`

Tips

Encode timestamps if your tooling requires it (: becomes %3A, + becomes %2B).
You can omit status to retrieve all jobs within a time range.
Jobs older than 30 days cannot be listed. Record the jobId of anything you need to look up later.

Note: Replace {index_name} with your actual index name, {api_key} with your API key, and {job_id} with the specific job ID you want to check.

Job Statuses

Jobs can have the following statuses:

Status	Description
`PENDING`	Job is queued and waiting to be processed
`IN_PROGRESS`	Job is currently being processed
`COMPLETED`	Job finished and every document was processed
`ERROR`	Job finished but some documents were rejected for a reason on your side, such as an image that could not be downloaded, an invalid URL, a field with the wrong type, or an update to an `_id` that is not in the index (see `errorItems` and `errorItemsDetails`)
`FAILED`	Job finished but some documents failed for a server-side reason (see `failedItems` and `failedItemsDetails`). Resend those documents later
`CONFLICT`	Job finished but some documents were skipped because a newer version was already indexed (see `conflictItems` and `conflictDetails`)
`CANCELLED`	Job was cancelled before completion

A finished job has exactly one status. When more than one applies, the precedence is FAILED > ERROR > CONFLICT > COMPLETED: a job with both server-side failures and rejected documents reports FAILED, and you should read errorItemsDetails as well as failedItemsDetails in that case.

Response Fields

List Jobs Response

Field	Type	Description
`jobs`	Array	List of job objects (most recent first), at most `limit` entries
`total`	Integer	Total number of jobs in the selected time window (may exceed the number returned under `jobs`)
`totalDocs`	Integer	Total number of documents currently in the index
`hasActiveJob`	Boolean	Whether there is any `PENDING` or `IN_PROGRESS` job in the returned `jobs`
`jobsByStatus`	Object	Count of jobs by status across the returned `jobs`
`docsByStatus`	Object	Document outcomes summed across the returned `jobs`, keyed by `PROCESSED`, `ERROR`, `FAILED`, `SKIPPED`, `PENDING` and, for cancelled jobs, `CANCELLED`
`timeFrom`	String	Start timestamp of the selected time window
`timeTo`	String	End timestamp of the selected time window

Job Object Fields

Field	Type	Description
`jobId`	String	Unique identifier for the job
`jobType`	String	How the job was created: `ECOM` for jobs created through the documents endpoints, `BULK`, `WEBHOOK` or `COLLECTION_PATCH` for jobs created by a platform integration such as Shopify
`jobStatus`	String	Current status of the job (see Job Statuses)
`platform`	String	`ecom` for jobs created through the documents endpoints, `shopify` for jobs created by the Shopify integration
`shopId`	String	Internal identifier of the index the job belongs to
`isInternal`	Boolean	`true` for jobs started by an internal Marqo process; jobs you create through the API are `false`
`initiatorUserId`	String/null	The user who started the job, if applicable
`retryCount`	Integer	Number of times the job has been retried
`platformData`	Object	Job metadata. `platformData.operation` is one of `ADD_DOCUMENTS`, `UPDATE_DOCUMENTS`, `DELETE_DOCUMENTS` or `UPDATE_DOCUMENTS_BY_SEARCH`; other keys depend on `jobType`
`totalItems`	Integer	Total number of items in the job
`processedItems`	Integer	Number of items processed successfully
`errorItems`	Integer	Number of items rejected for a reason on your side (drives the `ERROR` status)
`failedItems`	Integer	Number of items that failed for a server-side reason (drives the `FAILED` status)
`skippedItems`	Integer	Number of items skipped (non-error)
`conflictItems`	Integer	Number of items skipped due to version conflicts
`errorMessage`	String/null	General error message if the job as a whole failed
`errorItemsDetails`	Object	Map of rejection reasons to lists of item IDs rejected for that reason
`errorItemsSummary`	Object	Count of rejected items per reason
`errorItemDetails`	Object	Map of rejection reasons to the per-item error payload reported by the index for each rejected item, with `_id` and `reason` added. Empty for reasons detected before the document reached the index, such as `document_missing`
`failedItemsDetails`	Object	Map of failure reasons to lists of item IDs that failed for that reason
`errorSummary`	Object	Count of failed (server-side) items per reason. Despite the name this summarises `failedItemsDetails`, not `errorItemsDetails`; the name is kept for compatibility
`failedItemDetails`	Object	Map of failure reasons to per-item diagnostics for server-side failures
`failedPayloads`	Object	Map of item ID to the document that was sent and failed, so you can inspect or resend it
`skippedItemsDetails`	Object	Map of skip reasons to lists of skipped item IDs
`skippedSummary`	Object	Count of items per skip reason
`conflictDetails`	Object	Map of conflict reasons to conflict objects (e.g. conflicting `jobId`s)
`conflictSummary`	Object	Count of items per conflict reason
`createdAt`	String	ISO timestamp when job was created
`startedAt`	String/null	ISO timestamp when job started processing
`completedAt`	String/null	ISO timestamp when job completed
`progressPercentage`	Float	Completion percentage (0.0 to 100.0), counting processed, skipped and conflicting items
`estimatedTimeMinutes`	Integer/null	Estimated time for job completion

Response Examples

All Jobs Response

{
  "jobs": [
    {
      "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521",
      "jobType": "ECOM",
      "jobStatus": "COMPLETED",
      "platform": "ecom",
      "shopId": "your-index",
      "isInternal": false,
      "initiatorUserId": null,
      "retryCount": 0,
      "platformData": {
        "operation": "ADD_DOCUMENTS"
      },
      "totalItems": 3,
      "processedItems": 3,
      "failedItems": 0,
      "errorItems": 0,
      "skippedItems": 0,
      "conflictItems": 0,
      "errorMessage": null,
      "failedItemsDetails": {},
      "errorSummary": {},
      "errorItemsDetails": {},
      "errorItemDetails": {},
      "failedItemDetails": {},
      "errorItemsSummary": {},
      "skippedItemsDetails": {},
      "skippedSummary": {},
      "conflictDetails": {},
      "conflictSummary": {},
      "failedPayloads": {},
      "createdAt": "2025-08-12T14:50:22.272035+00:00",
      "startedAt": "2025-08-12T14:50:24.936806+00:00",
      "completedAt": "2025-08-12T14:50:35.980820+00:00",
      "progressPercentage": 100.0,
      "estimatedTimeMinutes": 1
    }
  ],
  "total": 1,
  "totalDocs": 1234,
  "hasActiveJob": false,
  "jobsByStatus": {
    "COMPLETED": 1
  },
  "docsByStatus": {
    "PROCESSED": 3,
    "FAILED": 0,
    "ERROR": 0,
    "SKIPPED": 0,
    "PENDING": 0
  },
  "timeFrom": "2025-08-11T14:50:22.272035+00:00",
  "timeTo": "2025-08-12T14:50:35.980820+00:00"
}

Specific Job Response

An update job in which one of three documents was rejected because its _id does not exist in the index:

{
  "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521",
  "jobType": "ECOM",
  "jobStatus": "ERROR",
  "platform": "ecom",
  "shopId": "your-index",
  "isInternal": false,
  "initiatorUserId": null,
  "retryCount": 0,
  "platformData": {
    "operation": "UPDATE_DOCUMENTS"
  },
  "totalItems": 3,
  "processedItems": 2,
  "failedItems": 0,
  "errorItems": 1,
  "skippedItems": 0,
  "conflictItems": 0,
  "errorMessage": null,
  "failedItemsDetails": {},
  "errorSummary": {},
  "errorItemsDetails": {
    "document_missing": ["43207430725"]
  },
  "errorItemDetails": {},
  "failedItemDetails": {},
  "errorItemsSummary": {
    "document_missing": 1
  },
  "skippedItemsDetails": {},
  "skippedSummary": {},
  "conflictDetails": {},
  "conflictSummary": {},
  "failedPayloads": {},
  "createdAt": "2025-08-12T14:50:22.272035+00:00",
  "startedAt": "2025-08-12T14:50:24.936806+00:00",
  "completedAt": "2025-08-12T14:50:35.980820+00:00",
  "progressPercentage": 66.0,
  "estimatedTimeMinutes": 1
}

The reason keys under errorItemsDetails and failedItemsDetails vary with the cause. document_missing (an update whose _id is not in the index) is one you will see for updates; most other keys are the error code reported by the index for that document. Treat them as descriptive strings rather than a fixed enumeration.

Workflow Example

Here's a typical workflow for adding products and monitoring the job:

1. Add Products (Returns Job ID)

curl -X POST https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/documents \
	-H "Authorization: Bearer {api_key}" \
	-H "Content-Type: application/json" \
	-d '{"documents": [
        {
            "_id": "light-blue-shirt",
            "parentProductId": "parent-shirt",
            "productTitle": "Shirt",
            "variantTitle": "Shirt - Light Blue",
            "variantImageUrl": "https://cdn.shopify.com/s/files/1/0705/8276/3687/files/light-blue-tshirt.jpg?v=1754569578",
            "price": 100,
            "color": "Light Blue",
            "productCollections": ["shirts"]
        }
      ]}'

Response (202 Accepted):

{
  "jobId": "4eb23b8c-8d50-4a39-9bb1-490d8e6df521"
}

2. Check Job Status

curl https://ecom.marqo-ep.ai/api/v1/indexes/{index_name}/jobs/4eb23b8c-8d50-4a39-9bb1-490d8e6df521 \
-H "Authorization: Bearer {api_key}"

3. Monitor Until Complete

Keep checking the job until jobStatus is no longer PENDING or IN_PROGRESS. COMPLETED means every document was processed. For ERROR, FAILED or CONFLICT, read the matching *Details fields for the affected document IDs and the reason.

Best Practices

Job Monitoring

Check Status Regularly: Monitor job progress, especially for large batches
Handle Failures: Check ERROR and FAILED jobs for details and resend the affected documents
Batch Appropriately: Keep document batches to 25-75 documents for optimal performance

Error Handling

Read the details: errorItemsDetails and failedItemsDetails name the documents and the reason; failedPayloads gives you the exact document that was sent
Retry Selectively: Fix and resubmit only the affected documents. Documents listed under failedItemsDetails can usually be resent unchanged; documents under errorItemsDetails need to be corrected first
Validate Before Submitting: Ensure documents meet all requirements

Prerequisites​

Understanding Asynchronous Jobs​

Job Operations​

List Jobs​

Get Specific Job​

Filter Jobs by Status​

Get Pending Jobs​

Get In Progress Jobs​

Get Jobs with Rejected Documents​

Get Failed Jobs​

Filter by Time Range​

Job Statuses​

Response Fields​

List Jobs Response​

Job Object Fields​

Response Examples​

All Jobs Response​

Specific Job Response​

Workflow Example​

1. Add Products (Returns Job ID)​

2. Check Job Status​

3. Monitor Until Complete​

Best Practices​

Job Monitoring​

Error Handling​