> For the complete documentation index, see [llms.txt](https://docs.midnight.network/llms.txt)

# Index contract state with EffectStream

Use this guide to build an indexer for a Midnight contract. An indexer watches the contract's public state, keeps its own records, and answers questions the chain does not.

The procedures target the `preview` network and a contract you need not deploy, so the whole guide runs with no local stack. Reading one chain is the smallest thing [EffectStream](https://effectstream.github.io/docs/) does; [Additional resources](#additional-resources) points to the cross-chain path.

Community-maintained, not officially supported

[EffectStream](https://github.com/effectstream/effectstream) maintains the `@effectstream/*` packages and the templates this guide uses, not the Midnight Foundation. The project changes often, so when its repository and this guide disagree, follow the repository. Report EffectStream problems on the [EffectStream tracker](https://github.com/effectstream/effectstream/issues), and problems with this page on the [Midnight docs repository](https://github.com/midnightntwrk/midnight-docs/issues).

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

These apply to every procedure in this guide:

* [Bun](https://bun.sh). EffectStream runs TypeScript directly and uses Bun as its package manager. See [Set up Bun for Midnight development](https://docs.midnight.network/how-to/bun-runtime-midnight.md).
* Network access to the hosted `preview` indexer listed in the [environment reference](https://docs.midnight.network/guides/networks-and-environments.md#environment-reference).
* `curl`, or any HTTP client, for the verification steps.

These procedures read from a hosted indexer over HTTPS, so they need no Docker, no Midnight node, no proof server, and no Compact toolchain.

Keep the pinned SDK version

This guide pins `@effectstream/node-sdk` to `0.104.3`, the version npm's `midnight-1` tag points to. That release builds on the ledger 8 packages in the [support matrix](https://docs.midnight.network/relnotes/support-matrix.md), such as Midnight.js 4.1.1. Preview, Preprod, and Mainnet now run ledger 9, and ledger 8 packages can't transact on them. The newest release on npm, the `0.200` line, depends on ledger 9 prereleases.

<!-- -->

## Why a DApp needs an indexer[​](#why-a-dapp-needs-an-indexer "Direct link to Why a DApp needs an indexer")

A Midnight contract stores its current state: its public ledger, the verifier keys for its circuits, its maintenance authority, and its token balances. Who holds the top score? What changed in the last hour? The chain answers neither. Midnight's own indexer can return a contract's state as of a past block, but it computes no totals and no per-field changes.

An indexer answers them. It watches state changes, applies a function you write to each one, and can store the result in a database you control. It handles changes in chain order, so replaying the same history produces the same result.

Midnight already runs an indexer, and this guide connects to it rather than replacing it. The Midnight indexer reports what the chain holds, and your DApp reaches it through the `publicDataProvider` in [Deploying and operating a contract](https://docs.midnight.network/guides/deploy-and-operate.md#the-transaction-pipeline-and-its-providers). EffectStream reads from that indexer and builds your own derived state on top.

Reading the ledger has two consequences. A change reaches your code only after several blocks of confirmation, so a chain reorganization cannot leave your database holding state the chain later drops. And your code sees exactly what the contract discloses, which makes [What the ledger schema can read](#what-the-ledger-schema-can-read) a contract decision.

<!-- -->

## Run the indexer[​](#run-the-indexer "Direct link to Run the indexer")

Build and run a node that watches a Midnight contract and prints each state change. The contract used here is a counter already running on `preview`, so you deploy nothing.

### Procedure[​](#procedure "Direct link to Procedure")

1. Create an empty project directory, then add a `package.json` with the single dependency an EffectStream node needs. Pin the version here rather than in the import statement:

   package.json

   ```
   {

     "name": "midnight-indexer",

     "private": true,

     "scripts": {

       "start": "bun index.ts"

     },

     "dependencies": {

       "@effectstream/node-sdk": "0.104.3"

     }

   }
   ```

2. Install with Bun. Run a real install rather than relying on the auto-install cache: the SDK reaches packages that import themselves by name, and resolving those imports needs a `node_modules` directory the cache never creates:

   ```
   bun install
   ```

3. Create `index.ts` alongside `package.json`. A single `runNode` call describes the whole node, taking a name, a database, the sources to watch, and one transition function per source:

   index.ts

   ```
   import { midnightContract, pglite, runNode } from "@effectstream/node-sdk";



   await runNode({

     appName: "midnight-indexer",

     database: pglite(),

     sources: {

       counter: midnightContract({

         network: "preview",

         address: "c1a9ec7c4d2566f59456fd915a0438bf4dc9b8671d4c2308d30af796c51ad20f",

         startBlockHeight: "latest",

         ledger: { round: "uint128" },

       }),

     },

     transitions: {

       counter: ({ state, blockHeight }) => {

         console.log(`round ${state.round} at block ${blockHeight}`);

       },

     },

   });
   ```

   `pglite()` gives the node an embedded PostgreSQL-compatible database, held in memory unless you pass a `dataDir`. The `sources` key, `counter` here, is yours to choose, and `transitions` reuses it so the two stay paired. Setting `network` selects the hosted indexer, and `address` takes the contract's 64-character hex address with no `0x` prefix. `startBlockHeight` takes `"latest"` to begin at the block the indexer is on, or a block number to replay from that height. [What the ledger schema can read](#what-the-ledger-schema-can-read) covers the `ledger` option, and whether it works for the contract you have in mind.

4. Start the node:

   ```
   bun start
   ```

5. Read the decoded state in the transition, which receives an object with `state`, `blockHeight`, `blockTimestamp`, `network`, and `address`. `blockHeight` is EffectStream's own block number, not the Midnight height. With the in-memory database it starts at 1 on every run. With a `dataDir`, the node reads the last block it recorded from the database and carries on from there. Unsigned integers arrive as decimal strings, because a `uint128` does not fit in a JavaScript number. Byte fields and map keys arrive as `0x`-prefixed hex strings. Convert with `BigInt(state.round)` when you need arithmetic.

Replay history instead of waiting for a call

A transition fires on state changes, not on blocks, so a quiet contract prints nothing. `startBlockHeight` takes a block number to replay from that height instead of starting at the tip. The node then fetches every block from that height to the tip, in batches of ten with one indexer request per block, before the first transition fires. Replay therefore only suits a call made recently. The hosted indexer answers `403 Forbidden` to a client that fetches too much, too fast, which a long replay does. Find the block of a contract's most recent call with the indexer's [`contractAction` query](https://docs.midnight.network/api-reference/midnight-indexer/operations/queries/contract-action.md). The counter used in this guide was last called at block 377864, which is too far behind the tip to replay. To watch a transition, point the node at a contract you can call yourself.

### Verification[​](#verification "Direct link to Verification")

The node connects to `preview`, starts its HTTP server, and logs each block it fetches, like this:

```
[MidnightClient] Using indexer https://indexer.preview.midnight.network/api/v4/graphql (network: preview)

Paima Engine HTTP server running on http://127.0.0.1:9999

[Midnight:preview] Fetching blocks from 788014 to 788014.

10:20:19 INFO effectstream-sync-block-merge: finalized block 1 @ undefined... | {"clock":[1,1]}

10:20:21 INFO effectstream-sync-block-merge: finalized block 3 @ undefined... | {"clock":[3,3],"midnight-counter":[788014,788014]}
```

The `finalized block` numbers are EffectStream's own clock, which starts at 1 on an empty database, and `@ undefined` fills the slot where a merged block hash would go. The Midnight height is the `midnight-counter` pair, which appears on the ticks that carried a Midnight block. Your heights differ.

A `round N at block M` line appears whenever the contract's state changes. A contract nobody is calling produces no such line, which is the case the tip above covers.

<!-- -->

## What the ledger schema can read[​](#what-the-ledger-schema-can-read "Direct link to What the ledger schema can read")

The `ledger` option declares how to decode public state, which is why a node needs no compiled contract artifacts. It also decides which contracts a node can read at all, so check it against your contract before you plan around it.

EffectStream matches schema keys to ledger fields by position. The first key reads the first field, the second key reads the second, and so on until your schema runs out. A schema accepts these types:

* `"uint8"` through `"uint128"`, decoded little-endian into decimal strings
* `"bytes"`, decoded into a `0x`-prefixed hex string
* `"boolean"`
* `{ type: "map", value: <type> }`, nesting arbitrarily, with keys as `0x` hex strings
* `{ type: "option", value: <type> }`, decoding to `null` when absent

A `List`, `MerkleTree`, or `HistoricMerkleTree` fails at parse time. A `struct`, `Vector`, or `enum` reads without error, but a schema cannot split a struct into named fields. Read a struct or `Vector` as `"bytes"` to get its raw bytes as one hex string, and an enum as `"uint8"`. Read a struct as a number and you get a wrong value, with no error. Because the match is positional, one unreadable field blocks every field after it. A `List` sitting third in a ledger costs you the fourth field and the fifth as well.

Some contracts need the compiler-generated decoder instead

A schema cannot read lists or Merkle trees, or split a struct into named fields. When your contract's ledger holds a list or tree near the top, or you need a struct's fields, use EffectStream's full configuration API instead. It takes the decoder the Compact compiler generates, which reads every exported ledger field. See the [EffectStream documentation](https://effectstream.github.io/docs/home/chains/midnight).

When you control the contract, you can design its ledger so a schema reads it. Put the fields an indexer needs at the top, and lists, Merkle trees, structs, and digests at the bottom. Flatten a struct that needs indexing into sibling top-level declarations keyed the same way. A `Map<Bytes<32>, GameState>` holding four small numbers becomes four `Map<Bytes<32>, Uint<8>>` declarations. The trade is four ledger containers and four writes per update instead of one, so weigh it against how often the values change.

```
// An indexer reads these, in this order.

export ledger status: Map<Bytes<32>, Uint<8>>;

export ledger winner: Map<Bytes<32>, Uint<8>>;

export ledger scores: Map<Bytes<32>, Map<Uint<16>, Uint<8>>>;



// Structs and digests go last.

export ledger commitments: Map<Bytes<32>, GameKeys>;
```

The matching schema stops where the structs begin:

```
ledger: {

  status: { type: "map", value: "uint8" },

  winner: { type: "map", value: "uint8" },

  scores: { type: "map", value: { type: "map", value: "uint8" } },

}
```

Ordering is an indexing decision, not a privacy one. Every ledger field is public wherever it sits, which is the point [Security and best practices](https://docs.midnight.network/guides/security-best-practices.md#on-chain-visibility) makes about `disclose()`. Put the fields a schema must read first. Identity commitments and Merkle roots serve in-circuit checks rather than readers, so they belong at the bottom.

<!-- -->

## Serve indexed state over HTTP[​](#serve-indexed-state-over-http "Direct link to Serve indexed state over HTTP")

Expose what the node records so the rest of your application can read it. `runNode` hosts an HTTP server.

### Prerequisites[​](#prerequisites-1 "Direct link to Prerequisites")

* A running node, from [Run the indexer](#run-the-indexer).

### Procedure[​](#procedure-1 "Direct link to Procedure")

1. Replace `index.ts` with this version. It records the value in a variable instead of logging it, and adds an `api` function to serve it:

   index.ts

   ```
   import { midnightContract, pglite, runNode } from "@effectstream/node-sdk";



   let round = "waiting for the next contract update";



   await runNode({

     appName: "midnight-indexer",

     database: pglite(),

     sources: {

       counter: midnightContract({

         network: "preview",

         address: "c1a9ec7c4d2566f59456fd915a0438bf4dc9b8671d4c2308d30af796c51ad20f",

         startBlockHeight: "latest",

         ledger: { round: "uint128" },

       }),

     },

     transitions: {

       counter: ({ state }) => {

         round = state.round;

       },

     },

     api: async (server) => {

       server.get("/round", async (_request, reply) => reply.send({ round }));

     },

   });
   ```

2. Restart the node and request the endpoint. The server listens on port 9999 unless you set `apiPort` on `runNode`.

Choose the database mode deliberately

`pglite()` holds data in memory by default, so every run starts at the chain tip and misses whatever happened while the node was down. Passing `pglite({ dataDir: "./data" })` stores the database on disk, so a restart can resume from the block height it recorded. With `0.104.3`, the first restart after the node creates the directory still starts at the tip, and later restarts resume. Confirm that your node resumes before you rely on it.

### Verification[​](#verification-1 "Direct link to Verification")

The endpoint answers with the value the last transition recorded, or the starting value when no transition has fired yet:

```
curl http://localhost:9999/round
```

```
{"round":"waiting for the next contract update"}
```

<!-- -->

## Indexer troubleshooting[​](#indexer-troubleshooting "Direct link to Indexer troubleshooting")

This table lists the failures you may hit on this guide's path, with their fixes.

| Symptom                                                                      | Cause                                                                                                    | Fix                                                                                                                                                            |
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Install fails resolving a package that imports itself by name                | Bun's auto-install cache created no `node_modules` anchor                                                | Run `bun install` before `bun start`                                                                                                                           |
| The first transition takes far longer than expected                          | EffectStream holds each change for several blocks of confirmation                                        | Wait. The delay keeps the database consistent with settled state                                                                                               |
| A comparison against a decoded number never matches                          | Unsigned integers decode to decimal strings, not numbers                                                 | Convert with `BigInt(state.field)` before comparing                                                                                                            |
| The node stops with `Unable to read the Midnight preview tip: 403 Forbidden` | The hosted indexer is refusing your client, which happens when a replay fetches too many blocks too fast | Wait before retrying, and replay only from a height close to the tip                                                                                           |
| Parse error naming an array where a scalar belongs                           | A `List`, `MerkleTree`, or `HistoricMerkleTree` sits at a schema position                                | Use the compiler-generated decoder, or reorder the ledger if you control the contract; see [What the ledger schema can read](#what-the-ledger-schema-can-read) |

## Additional resources[​](#additional-resources "Direct link to Additional resources")

* [EffectStream documentation](https://effectstream.github.io/docs/): the project's own documentation, and the cross-chain templates that pair Midnight with an EVM chain, Bitcoin, or Cardano.
* [Deploying and operating a contract](https://docs.midnight.network/guides/deploy-and-operate.md): the providers a DApp uses to reach the Midnight indexer directly, and how to deploy a contract of your own to index.
* [Networks and environments](https://docs.midnight.network/guides/networks-and-environments.md): every endpoint, network ID, and the local stack.
* [Security and best practices](https://docs.midnight.network/guides/security-best-practices.md): what a chain observer sees, and how `disclose()` decides it.
* [Support matrix](https://docs.midnight.network/relnotes/support-matrix.md): the Midnight versions each network runs.
