Skip to main content
For the complete documentation index, see llms.txt

Proof server errors

The proof server is a standalone HTTP service that generates zero-knowledge proofs for transactions. It is part of the midnight-ledger repository and typically runs on port 6300.

It is the only crate in midnight-ledger that uses thiserror derives for error handling. The server maps errors to HTTP status codes and returns them as HTTP responses.

Proof server versions

After the ledger 9 network upgrade, Mainnet, Preview, and Preprod use the ledger 9 proof server, version 9.0.2. Proof server 8.1.3 served Preprod until its upgrade on 9 November 2026. Both versions return the same errors and status codes. The main difference is the default job capacity: proof server 9 limits the queue to 10 pending jobs by default, so 429 Too Many Requests can occur without any configuration. Proof server 8.1.3 has no limit by default.

Worker pool errors (WorkerPoolError)​

These errors arise from the proof server's internal job queue and worker pool management.

HTTP statusErrorDescriptionFix
400 Bad RequestJobNotPending(Uuid)Tried to cancel a non-pending job.Job is already processing, completed, or cancelled.
428 Precondition RequiredJobMissing(Uuid)Referenced job not found.The job ID is invalid or has expired.
429 Too Many RequestsJobQueueFullThe number of pending jobs has reached the job capacity. Proof server 9 sets the capacity to 10 by default. Proof server 8.1.3 sets it to 0 (no limit) by default, so this error occurs only when you set a capacity.Wait and retry with backoff, or raise the capacity with --job-capacity or MIDNIGHT_PROOF_SERVER_JOB_CAPACITY.
500 Internal Server ErrorChannelClosedInternal work channel closed.Restart the proof server.

Work errors (WorkError)​

These errors occur during the actual proof generation process.

HTTP StatusErrorDescriptionFix
400 Bad RequestBadInput(String)Proof request input data is invalid. In proof server 9, the error message includes the detail string.Check the transaction data being sent for proving.
500 Internal Server ErrorCancelledUnexpectedlyThe system cancelled the job without an explicit requestInternal error; retry the proof request
500 Internal Server ErrorInternalError(String)Internal proof generation error.Check proof server logs; restart the proof server if the error persists.
500 Internal Server ErrorJoinErrorTask join error during proof generation.Internal threading error; retry or restart.

Job status enum​

The proof server tracks each proof job through the following states.

StatusDescription
PendingJob queued, waiting for a worker.
ProcessingProof generation in progress.
CancelledThe system cancelled this job.
Error(WorkError)Job failed with an error.
Success(Vec<u8>)Proof generated successfully.

Health endpoint​

Use the /ready endpoint to check whether the proof server is accepting new requests. The JSON response reports status, jobsProcessing, jobsPending, jobCapacity, and timestamp.

EndpointHTTP statusDescription
GET /ready200 OKServer is ready to accept proof requests.
GET /ready503 Service UnavailableServer is busy: the number of pending jobs has reached the job capacity. A new proof request gets 429 JobQueueFull. With no capacity set (the 8.1.3 default), /ready doesn't return 503.

Common issues​

The following table covers the most frequent problems and their likely causes.

SymptomLikely causeFix
429 responsesPending proof requests have filled the job queue. Proof server 9 allows 10 pending jobs by default.Reduce the number of parallel proof requests and implement an exponential backoff retry strategy so requests are spaced out under load. If you run your own proof server, raise --job-capacity.
503 from /readyThe job queue is full.Wait for current proofs to complete, raise --job-capacity, or add workers with --num-workers.
400 with "bad input"Malformed transaction dataVerify the transaction was built correctly with the SDK.
Connection refused on port 6300Proof server not runningStart the proof server container, check Docker status.
Slow proof generationLarge circuit or insufficient resourcesAllocate more CPU or memory to the proof server container.