Version: v1

For the complete documentation index, see llms.txt

Index contract state with EffectStream

Use this guide to build an indexer for a Midnight contract. An indexer watches the contract's public state, keeps its own records, and answers questions the chain does not.

The procedures target the preview network and a contract you need not deploy, so the whole guide runs with no local stack. Reading one chain is the smallest thing EffectStream does; Additional resources points to the cross-chain path.

Community-maintained, not officially supported

EffectStream maintains the @effectstream/* packages and the templates this guide uses, not the Midnight Foundation. The project changes often, so when its repository and this guide disagree, follow the repository. Report EffectStream problems on the EffectStream tracker, and problems with this page on the Midnight docs repository.

Prerequisites

These apply to every procedure in this guide:

Bun. EffectStream runs TypeScript directly and uses Bun as its package manager. See Set up Bun for Midnight development.
Network access to the hosted preview indexer listed in the environment reference.
curl, or any HTTP client, for the verification steps.

These procedures read from a hosted indexer over HTTPS, so they need no Docker, no Midnight node, no proof server, and no Compact toolchain.

The compatibility matrix does not cover EffectStream

The EffectStream packages depend on no @midnight-ntwrk/* package, so the support matrix governs the contract half of a project and not this half.

Why a DApp needs an indexer

A Midnight contract stores its current public state and nothing else. Who holds the top score? What changed in the last hour? The chain answers neither. It keeps no history you can query, and it computes no totals.

An indexer answers them. It watches state changes, applies a function you write to each one, and can store the result in a database you control. It handles changes in chain order, so replaying the same history produces the same result.

Midnight already runs an indexer, and this guide connects to it rather than replacing it. The Midnight indexer reports what the chain holds, and your DApp reaches it through the publicDataProvider in Deploying and operating a contract. EffectStream reads from that indexer and builds your own derived state on top.

Reading the ledger has two consequences. A change reaches your code only after several blocks of confirmation, so a chain reorganization cannot leave your database holding state the chain later drops. And your code sees exactly what the contract discloses, which makes What the ledger schema can read a contract decision.

Run the indexer

Build and run a node that watches a Midnight contract and prints each state change. The contract used here is a counter already running on preview, so you deploy nothing.

Procedure

Create an empty project directory, then add a package.json with the single dependency an EffectStream node needs. Pin the version here rather than in the import statement:

package.json
{
  "name": "midnight-indexer",
  "private": true,
  "scripts": {
    "start": "bun index.ts"
  },
  "dependencies": {
    "@effectstream/node-sdk": "0.104.0"
  }
}

Install with Bun. Run a real install rather than relying on the auto-install cache: the SDK reaches packages that import themselves by name, and resolving those imports needs a node_modules directory the cache never creates:
```
bun install
```
Create index.ts alongside package.json. A single runNode call describes the whole node, taking a name, a database, the sources to watch, and one transition function per source:
index.ts
```
import { midnightContract, pglite, runNode } from "@effectstream/node-sdk";

await runNode({
  appName: "midnight-indexer",
  database: pglite(),
  sources: {
    counter: midnightContract({
      network: "preview",
      address: "c1a9ec7c4d2566f59456fd915a0438bf4dc9b8671d4c2308d30af796c51ad20f",
      startBlockHeight: "latest",
      ledger: { round: "uint128" },
    }),
  },
  transitions: {
    counter: ({ state, blockHeight }) => {
      console.log(`round ${state.round} at block ${blockHeight}`);
    },
  },
});
```
pglite() gives the node an embedded PostgreSQL-compatible database, held in memory unless you pass a dataDir. The sources key, counter here, is yours to choose, and transitions reuses it so the two stay paired. Setting network selects the hosted indexer, and address takes the contract's 64-character hex address with no 0x prefix. startBlockHeight takes "latest" to begin at the block the indexer is on, or a block number to replay from that height. What the ledger schema can read covers the ledger option, and whether it works for the contract you have in mind.
Start the node:
```
bun start
```
Read the decoded state in the transition, which receives an object with state, blockHeight, blockTimestamp, network, and address. Unsigned integers arrive as decimal strings, because a uint128 does not fit in a JavaScript number. Byte fields and map keys arrive as 0x-prefixed hex strings. Convert with BigInt(state.round) when you need arithmetic.

Replay history instead of waiting for a call

A transition fires on state changes, not on blocks, so a quiet contract prints nothing. startBlockHeight takes a block number to replay from that height instead of starting at the tip. Keep the replay short: the node fetches a few blocks per second, and the hosted indexer answers 403 Forbidden to a client that fetches too much, too fast. Find the block of the contract's most recent call with the indexer's contractAction query and start a little below it. The counter used in this guide was last called at block 377864.

Verification

The node connects to preview, starts its HTTP server, and logs each block it fetches, like this:

[MidnightClient] Using indexer https://indexer.preview.midnight.network/api/v4/graphql (network: preview)
Paima Engine HTTP server running on http://127.0.0.1:9999
[Midnight:preview] Fetching blocks from 788014 to 788014.
10:20:19 INFO effectstream-sync-block-merge: finalized block 1 @ undefined... | {"clock":[1,1]}
10:20:21 INFO effectstream-sync-block-merge: finalized block 3 @ undefined... | {"clock":[3,3],"midnight-counter":[788014,788014]}

The finalized block numbers are EffectStream's own clock, which starts at 1 on every run, and @ undefined fills the slot where a merged block hash would go. The Midnight height is the midnight-counter pair, which appears on the ticks that carried a Midnight block. Your heights differ.

A round N at block M line appears whenever the contract's state changes. A contract nobody is calling produces no such line, which is the case the tip above covers.

What the ledger schema can read

The ledger option declares how to decode public state, which is why a node needs no compiled contract artifacts. It also decides which contracts a node can read at all, so check it against your contract before you plan around it.

EffectStream matches schema keys to ledger fields by position. The first key reads the first field, the second key reads the second, and so on until your schema runs out. A schema accepts these types:

"uint8" through "uint128", decoded little-endian into decimal strings
"bytes", decoded into a 0x-prefixed hex string
"boolean"
{ type: "map", value: <type> }, nesting arbitrarily, with keys as 0x hex strings
{ type: "option", value: <type> }, decoding to null when absent

A struct, Vector, or enum fails at parse time. Because the match is positional, one unreadable field blocks every field after it. A struct sitting third in a ledger costs you the fourth field and the fifth as well.

Some contracts need the compiler-generated decoder instead

A schema cannot express structs, enums, or wrapped types, and Compact contracts use all three. When your contract's ledger holds one near the top, use EffectStream's full configuration API instead. It takes the decoder the Compact compiler generates and reads anything the language can express. See the EffectStream documentation.

When you control the contract, you can design its ledger so a schema reads it. Put the fields an indexer needs at the top, and structs and digests at the bottom. Flatten a struct that needs indexing into sibling top-level declarations keyed the same way. A Map<Bytes<32>, GameState> holding four small numbers becomes four Map<Bytes<32>, Uint<8>> declarations. The trade is four ledger containers and four writes per update instead of one, so weigh it against how often the values change.

// An indexer reads these, in this order.
export ledger status: Map<Bytes<32>, Uint<8>>;
export ledger winner: Map<Bytes<32>, Uint<8>>;
export ledger scores: Map<Bytes<32>, Map<Uint<16>, Uint<8>>>;

// Structs and digests go last.
export ledger commitments: Map<Bytes<32>, GameKeys>;

The matching schema stops where the structs begin:

ledger: {
  status: { type: "map", value: "uint8" },
  winner: { type: "map", value: "uint8" },
  scores: { type: "map", value: { type: "map", value: "uint8" } },
}

Ordering is an indexing decision, not a privacy one. Every ledger field is public wherever it sits, which is the point Security and best practices makes about disclose(). Put the fields a schema must read first. Identity commitments and Merkle roots serve in-circuit checks rather than readers, so they belong at the bottom.

Serve indexed state over HTTP

Expose what the node records so the rest of your application can read it. runNode hosts an HTTP server.

Prerequisites

A running node, from Run the indexer.

Procedure

Replace index.ts with this version. It records the value in a variable instead of logging it, and adds an api function to serve it:

index.ts
import { midnightContract, pglite, runNode } from "@effectstream/node-sdk";

let round = "waiting for the next contract update";

await runNode({
  appName: "midnight-indexer",
  database: pglite(),
  sources: {
    counter: midnightContract({
      network: "preview",
      address: "c1a9ec7c4d2566f59456fd915a0438bf4dc9b8671d4c2308d30af796c51ad20f",
      startBlockHeight: "latest",
      ledger: { round: "uint128" },
    }),
  },
  transitions: {
    counter: ({ state }) => {
      round = state.round;
    },
  },
  api: async (server) => {
    server.get("/round", async (_request, reply) => reply.send({ round }));
  },
});

Restart the node and request the endpoint. The server listens on port 9999 unless you set apiPort on runNode.

Choose the database mode deliberately

pglite() holds data in memory by default, so every run starts at the chain tip and misses whatever happened while the node was down. Passing pglite({ dataDir: "./data" }) persists the database, and the node then resumes from the block height it recorded.

Verification

The endpoint answers with the value the last transition recorded, or the starting value when no transition has fired yet:

curl http://localhost:9999/round

{"round":"waiting for the next contract update"}

Indexer troubleshooting

This table lists the failures you may hit on this guide's path, with their fixes.

Symptom	Cause	Fix
Install fails resolving a package that imports itself by name	Bun's auto-install cache created no `node_modules` anchor	Run `bun install` before `bun start`
The first transition takes far longer than expected	EffectStream holds each change for several blocks of confirmation	Wait. The delay keeps the database consistent with settled state
A comparison against a decoded number never matches	Unsigned integers decode to decimal strings, not numbers	Convert with `BigInt(state.field)` before comparing
The node stops with `Unable to read the Midnight preview tip: 403 Forbidden`	The hosted indexer is refusing your client, which happens when a replay fetches too many blocks too fast	Wait before retrying, and start the replay a little below the height you need rather than at block 1
Parse error naming an array where a scalar belongs	A `struct`, `enum`, or `Vector` sits at a schema position	Use the compiler-generated decoder, or reorder the ledger if you control the contract; see What the ledger schema can read

Additional resources

EffectStream documentation: the project's own documentation, and the cross-chain templates that pair Midnight with an EVM chain, Bitcoin, or Cardano.
Deploying and operating a contract: the providers a DApp uses to reach the Midnight indexer directly, and how to deploy a contract of your own to index.
Networks and environments: every endpoint, network ID, and the local stack.
Security and best practices: what a chain observer sees, and how disclose() decides it.
Support matrix: the versions governing the contract half of a project.

Prerequisites​

Why a DApp needs an indexer​

Run the indexer​

Procedure​

Verification​

What the ledger schema can read​

Serve indexed state over HTTP​

Prerequisites​

Procedure​

Verification​

Indexer troubleshooting​

Additional resources​

Prerequisites

Why a DApp needs an indexer

Run the indexer

Procedure

Verification

What the ledger schema can read

Serve indexed state over HTTP

Prerequisites

Procedure

Verification

Indexer troubleshooting

Additional resources