cometindex 1.3.1

A library for building indexers for CometBFT events
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127

# pindexer

To run `pindexer`, you will need to create two postgres databases: one for the raw data and one for the compiled data. The raw data is the data from CometBFT, while the compiled data is translated into schemas determined by `pindexer`.

`pindexer` will read from the raw data database and write to the compiled data database, but doing so also requires a local database of ABCI events to be present. The most straightforward way to create such a database is by following the devnet instructions in the [Penumbra Guide].

[Penumbra Guide]: https://guide.penumbra.zone/dev/devnet-quickstart.html

# macos 

```
brew install postgresql@16
```
start
```
brew services start postgresql@16
```
create database `testnet_raw`
```
$ psql postgres
psql (16.3 (Homebrew))
Type "help" for help.

postgres=# CREATE DATABASE testnet_raw;
CREATE DATABASE
postgres=#
\q
```
test connection
```
psql "postgresql://localhost:5432/testnet_raw?sslmode=disable"
```
put in comet config.toml
```
[tx_index]
indexer = "psql"
psql-conn = "postgresql://localhost:5432/testnet_raw?sslmode=disable"
```

example data
```
testnet_raw=# SELECT * FROM events JOIN blocks ON events.block_id = blocks.rowid WHERE blocks.height = 426939 ;
 rowid | block_id | tx_id |                            type                             | rowid | height |         chain_id          |          created_at
-------+----------+-------+-------------------------------------------------------------+-------+--------+---------------------------+-------------------------------
 99043 |     7534 |       | block                                                       |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99044 |     7534 |       | penumbra.core.component.sct.v1.EventAnchor                  |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99045 |     7534 |       | penumbra.core.component.sct.v1.EventBlockRoot               |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99046 |     7534 |  4350 | tx                                                          |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99047 |     7534 |  4350 | tx                                                          |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99048 |     7534 |  4350 | penumbra.core.component.shielded_pool.v1.EventSpend         |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99049 |     7534 |  4350 | penumbra.core.component.sct.v1.EventCommitment              |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99050 |     7534 |  4350 | penumbra.core.component.shielded_pool.v1.EventOutput        |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99051 |     7534 |  4350 | penumbra.core.component.sct.v1.EventCommitment              |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99052 |     7534 |  4350 | penumbra.core.component.shielded_pool.v1.EventOutput        |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99053 |     7534 |  4350 | penumbra.core.component.shielded_pool.v1.EventBroadcastClue |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99054 |     7534 |  4350 | penumbra.core.component.shielded_pool.v1.EventBroadcastClue |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99055 |     7534 |  4351 | tx                                                          |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99056 |     7534 |  4351 | tx                                                          |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99057 |     7534 |  4351 | penumbra.core.component.shielded_pool.v1.EventSpend         |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99058 |     7534 |  4351 | penumbra.core.component.sct.v1.EventCommitment              |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99059 |     7534 |  4351 | penumbra.core.component.shielded_pool.v1.EventOutput        |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99060 |     7534 |  4351 | penumbra.core.component.sct.v1.EventCommitment              |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99061 |     7534 |  4351 | penumbra.core.component.shielded_pool.v1.EventOutput        |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99062 |     7534 |  4351 | penumbra.core.component.shielded_pool.v1.EventBroadcastClue |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
 99063 |     7534 |  4351 | penumbra.core.component.shielded_pool.v1.EventBroadcastClue |  7534 | 426939 | penumbra-testnet-deimos-8 | 2024-06-07 01:38:33.440578-04
(21 rows)
```

example invocations
```
cargo run --bin pindexer -- -s "postgresql://localhost:5432/testnet_raw?sslmode=disable" -d "postgresql://localhost:5432/testnet_compiled?sslmode=disable"
```
```
cargo run --example fmd_clues -- -s "postgresql://localhost:5432/testnet_raw?sslmode=disable" -d "postgresql://localhost:5432/testnet_compiled?sslmode=disable"
```

# nixos

initialize postgres

```
[user@hostname:~/penumbra]$ initdb --pgdata=.data
```

create a directory in `/run/postgresql`, or alter the generated configuration 
file and change the value of `unix_socket_directories`.

start psql

```
pg_ctl -D .data -l logfile start
```

create database `testnet_raw` and `testnet_compiled`

```
$ psql postgres

postgres=# CREATE DATABASE testnet_raw;
CREATE DATABASE

postgres=# CREATE DATABASE testnet_compiled;
CREATE DATABASE

postgres=# \q
```

test connection

```
psql "postgresql://localhost:5432/testnet_raw?sslmode=disable"
```

put in comet config.toml

```
# ~/.penumbra/network_data/node0/cometbft/config/config.toml
[tx_index]
indexer = "psql"
psql-conn = "postgresql://localhost:5432/testnet_raw?sslmode=disable"
```

install comet schema

```
psql --file=crates/util/cometindex/vendor/schema.sql "postgresql://localhost:5432/testnet_raw?sslmode=disable"
```