> For the complete documentation index, see [llms.txt](https://docs.midnight.network/llms.txt)

# Proof server errors

The proof server is a standalone HTTP service that generates zero-knowledge proofs for transactions. It is part of the `midnight-ledger` repository and typically runs on port 6300.

It is the only crate in `midnight-ledger` that uses `thiserror` derives for error handling. The server maps errors to HTTP status codes and returns them as HTTP responses.

Proof server versions

After the [ledger 9 network upgrade](https://docs.midnight.network/relnotes/ledger-9.md), Mainnet, Preview, and Preprod use the ledger 9 proof server, version 9.0.0. Proof server 8.1.3 served Preprod until its upgrade on 9 November 2026. Both versions return the same errors and status codes. The main difference is the default job capacity: proof server 9 limits the queue to 10 pending jobs by default, so `429 Too Many Requests` can occur without any configuration. Proof server 8.1.3 has no limit by default.

## Worker pool errors (`WorkerPoolError`)[​](#worker-pool-errors-workerpoolerror "Direct link to worker-pool-errors-workerpoolerror")

These errors arise from the proof server's internal job queue and worker pool management.

| HTTP status               | Error                 | Description                                                                                                                                                                                                             | Fix                                                                                                               |
| ------------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| 400 Bad Request           | `JobNotPending(Uuid)` | Tried to cancel a non-pending job.                                                                                                                                                                                      | Job is already processing, completed, or cancelled.                                                               |
| 428 Precondition Required | `JobMissing(Uuid)`    | Referenced job not found.                                                                                                                                                                                               | The job ID is invalid or has expired.                                                                             |
| 429 Too Many Requests     | `JobQueueFull`        | The number of pending jobs has reached the job capacity. Proof server 9 sets the capacity to 10 by default. Proof server 8.1.3 sets it to `0` (no limit) by default, so this error occurs only when you set a capacity. | Wait and retry with backoff, or raise the capacity with `--job-capacity` or `MIDNIGHT_PROOF_SERVER_JOB_CAPACITY`. |
| 500 Internal Server Error | `ChannelClosed`       | Internal work channel closed.                                                                                                                                                                                           | Restart the proof server.                                                                                         |

## Work errors (`WorkError`)[​](#work-errors-workerror "Direct link to work-errors-workerror")

These errors occur during the actual proof generation process.

| HTTP Status               | Error                   | Description                                                                                           | Fix                                                                      |
| ------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| 400 Bad Request           | `BadInput(String)`      | Proof request input data is invalid. In proof server 9, the error message includes the detail string. | Check the transaction data being sent for proving.                       |
| 500 Internal Server Error | `CancelledUnexpectedly` | The system cancelled the job without an explicit request                                              | Internal error; retry the proof request                                  |
| 500 Internal Server Error | `InternalError(String)` | Internal proof generation error.                                                                      | Check proof server logs; restart the proof server if the error persists. |
| 500 Internal Server Error | `JoinError`             | Task join error during proof generation.                                                              | Internal threading error; retry or restart.                              |

## Job status enum[​](#job-status-enum "Direct link to Job status enum")

The proof server tracks each proof job through the following states.

| Status             | Description                       |
| ------------------ | --------------------------------- |
| `Pending`          | Job queued, waiting for a worker. |
| `Processing`       | Proof generation in progress.     |
| `Cancelled`        | The system cancelled this job.    |
| `Error(WorkError)` | Job failed with an error.         |
| `Success(Vec<u8>)` | Proof generated successfully.     |

## Health endpoint[​](#health-endpoint "Direct link to Health endpoint")

Use the `/ready` endpoint to check whether the proof server is accepting new requests. The JSON response reports `status`, `jobsProcessing`, `jobsPending`, `jobCapacity`, and `timestamp`.

| Endpoint     | HTTP status             | Description                                                                                                                                                                                  |
| ------------ | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /ready` | 200 OK                  | Server is ready to accept proof requests.                                                                                                                                                    |
| `GET /ready` | 503 Service Unavailable | Server is busy: the number of pending jobs has reached the job capacity. A new proof request gets `429 JobQueueFull`. With no capacity set (the 8.1.3 default), `/ready` doesn't return 503. |

## Common issues[​](#common-issues "Direct link to Common issues")

The following table covers the most frequent problems and their likely causes.

| Symptom                         | Likely cause                                                                                        | Fix                                                                                                                                                                                               |
| ------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 429 responses                   | Pending proof requests have filled the job queue. Proof server 9 allows 10 pending jobs by default. | Reduce the number of parallel proof requests and implement an exponential backoff retry strategy so requests are spaced out under load. If you run your own proof server, raise `--job-capacity`. |
| 503 from `/ready`               | The job queue is full.                                                                              | Wait for current proofs to complete, raise `--job-capacity`, or add workers with `--num-workers`.                                                                                                 |
| 400 with "bad input"            | Malformed transaction data                                                                          | Verify the transaction was built correctly with the SDK.                                                                                                                                          |
| Connection refused on port 6300 | Proof server not running                                                                            | Start the proof server container, check Docker status.                                                                                                                                            |
| Slow proof generation           | Large circuit or insufficient resources                                                             | Allocate more CPU or memory to the proof server container.                                                                                                                                        |
