calimero-server 0.10.0

Core Calimero infrastructure and tools
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
# Node Server

- [Node Server]#node-server
  - [Introduction]#introduction
  - [Build Modes]#build-modes
    - [Default mode (external auth)]#default-mode-external-auth
    - [Embedded auth mode]#embedded-auth-mode
    - [1. Admin API]#1-admin-api
    - [2. JSON rpc]#2-json-rpc
      - [Query Method]#query-method
      - [Mutate Method]#mutate-method
    - [3. Websocket]#3-websocket
      - [Subscription Handling:]#subscription-handling
      - [Unsubscription Handling:]#unsubscription-handling
    - [4. Server Sent Event (SSE)]#4-server-sent-event-sse
      - [SSE Subscription Handling:]#sse-subscription-handling
      - [SSE Unsubscription Handling:]#sse-unsubscription-handling
  - [Node Server Workflows]#node-server-workflows
    - [Client Login Workflow]#client-login-workflow
    - [JSON rpc Workflow]#json-rpc-workflow
    - [Websocket Workflow]#websocket-workflow
    - [SSE Workflow]#sse-workflow
  - [Admin API endpoints]#admin-api-endpoints
    - [Protected Routes]#protected-routes
    - [Unprotected Routes]#unprotected-routes
  - [JSON rpc endpoint]#json-rpc-endpoint
  - [Websocket endpoints]#websocket-endpoints
  - [SSE endpoints]#sse-endpoints
  - [Examples]#examples

## Introduction

Node Server is a component in node that facilitates node administration and
enables communication with the logic of an application (loaded wasm) in
participating contexts.

Node Server component is split into 4 parts:

1.  [Admin API]https://github.com/calimero-network/core/blob/feat-admin_api_docs/crates/server/src/admin/service.rs
2.  [JSON rpc]https://github.com/calimero-network/core/blob/feat-admin_api_docs/crates/server/src/jsonrpc.rs
3.  [Websocket]https://github.com/calimero-network/core/blob/feat-admin_api_docs/crates/server/src/ws.rs
4.  [SSE]https://github.com/calimero-network/core/blob/master/crates/server/src/sse.rs

## Build Modes

The node server supports two authentication modes that can be selected at runtime:

### Default mode (external auth)

- The node expects an external `mero-auth` deployment and continues to forward
  authentication traffic to it, matching the current production setup.
- Existing configuration files work without modification.

### Embedded auth mode

- Choose the mode at runtime: `merod run --auth-mode embedded` or set
  `network.server.auth_mode = "embedded"` in `config.toml`.
- The node boots the embedded `mero-auth` service, mounts its `/auth` and `/admin`
  routes, and guards the JSON-RPC, admin, WebSocket, and SSE endpoints with the
  bundled JWT middleware.
- When `auth_mode = "embedded"` but there is no `[network.server.embedded_auth]`
  block, the service defaults to a local RocksDB store under the node's home
  directory (`auth/`) and enables only the `user_password` provider.
- `merod init --auth-mode embedded` writes those defaults during node
  initialization; use `--auth-storage-path <dir>` to move the RocksDB data or
  `--auth-storage memory` for an ephemeral setup.
- Switching back to proxy mode is as simple as running with
  `--auth-mode proxy` (or removing the `auth_mode` line), keeping existing
  deployments compatible.

#### Configuring `server.embedded_auth`

When embedded mode is active you can inline authentication settings under
`network.server.embedded_auth`. The schema matches `mero-auth` configuration; for example:

```toml
[network.server.embedded_auth]
listen_addr = "0.0.0.0:3001"

[network.server.embedded_auth.jwt]
issuer = "calimero-auth"
access_token_expiry = 3600
refresh_token_expiry = 2_592_000

[network.server.embedded_auth.storage]
type = "rocksdb"
path = "data/auth"

[network.server.embedded_auth.providers]
user_password = true
```

Omitting the block leaves the embedded defaults in place (RocksDB storage at
`auth/`, username/password authentication). External deployments can keep existing
configs unchanged while opting in environment by environment.

#### `merod init` with embedded auth

To initialize a node with embedded authentication, use the `--auth-mode embedded` flag:

```bash
merod --node my-node init --auth-mode embedded
```

By default, this will configure RocksDB storage at `auth/` within the node's home directory.
You can customize the storage:

- `--auth-storage-path <PATH>`: Specify a custom path for RocksDB storage.
- `--auth-storage memory`: Use in-memory storage (for testing, not persistent).

Example with in-memory storage:

```bash
merod --node my-node init --auth-mode embedded --auth-storage memory
```

#### CI/CD expectations

- Build the node once: `cargo build -p calimero-server`.
- Publish or package the single artefact; operators select the mode through
  configuration or the CLI flag.
- For the embedded auth build, ensure the environment exposes the frontend assets:
  - `CALIMERO_AUTH_FRONTEND_SRC` pointing to a release archive or local build, or
  - `CALIMERO_AUTH_FRONTEND_PATH` pointing to a prebuilt directory.
  - When pulling from GitHub, set `CALIMERO_AUTH_FRONTEND_ASSET` to use a
    release asset; otherwise the build downloads a source archive from the tag
    in `CALIMERO_AUTH_FRONTEND_VERSION`. If `CALIMERO_AUTH_FRONTEND_VERSION` is
    `latest`, it resolves the latest release tag via the GitHub release redirect
    and falls back to the branch in `CALIMERO_AUTH_FRONTEND_REF` (defaults to
    `master`) if the tag cannot be determined.
- When running integration suites, exercise at least one smoke test in each mode to
  confirm `/auth` endpoints are reachable in embedded mode and that the proxy mode
  continues to rely on the external service.

### 1. Admin API

The Admin API component of the Node Server exposes API for connection with the
node and its functionalities. It is primarily utilized by the Admin Dashboard to
query and manage various aspects of the node, including:

- Identity information
- Root keys
- Client keys
- Installed applications
- Started contexts

**Data Querying**: The Admin API allows the Admin Dashboard to fetch important
data from the node, such as identity details, root and client keys, and
information about installed and active applications.

**Application Management**: The API provides functionalities for managing
applications and contexts, allowing for the installation and uninstallation of
applications and starting contexts.

**Key Management**: Administrators can manage root and client keys through the
API, ensuring secure access and control over the node.

**Authentication**: The Admin API facilitates user authentication via selected
wallets, currently supporting MetaMask and NEAR networks. Authentication details
will be explained in later sections.

**Integration with Web Applications**: The authentication mechanism is also used
by web applications designed for applications (loaded wasm) in participating
context, ensuring secure and authenticated access.

### 2. JSON rpc

The JSON rpc component of the Node Server facilitates communication between the
clients and the context. This allows seamless interaction and data management
for applications.

The JSON rpc interface provides two primary methods:

- Query
- Mutate

#### Query Method

The `Query` method retrieves data from the application in participating context.
For instance, in the Only Peers forum application, posts and comments stored in
the application's storage can be queried using the JSON rpc interface. This
enables users to fetch and display content from the forum.

#### Mutate Method

The `Mutate` method allows modification of the application's data in
participating context. For example, in the Only Peers forum application, users
can create new posts or comments. The Mutate method updates the application's
storage with these new entries, facilitating dynamic content creation and
interaction within the application.

### 3. Websocket

The WebSocket is used for subscribing to and unsubscribing from certain context
running in the Node Server. Defined handlers manage subscription states for
WebSocket connections, allowing clients to receive updates about specific
contexts they are interested in. WebSocket handlers are essential for managing
real-time subscriptions within the Node Server. They allow clients to
dynamically subscribe to and unsubscribe from updates about various application
contexts.

**Important**: WebSocket connections are **unidirectional for events** - the server
pushes events to subscribed clients. For executing transactions (`mutate`) or 
reading state (`query`), clients must use the separate JSON-RPC endpoint. This 
separation of concerns provides:

- **WebSocket/SSE**: Real-time event streaming (Server → Client)
- **JSON-RPC**: Execute transactions and queries (Client ↔ Server)

This architecture ensures efficient event streaming while maintaining reliable
request/response patterns for state modifications.

#### Subscription Handling:

Websocket handles requests to subscribe to specific contexts and send responses
back to the client with the subscribed context IDs.

#### Unsubscription Handling:

Websocket handle requests to unsubscribe from specific contexts and send
responses back to the client with the unsubscribed context IDs.

### 4. Server Sent Event (SSE)

The Server-Sent Events (SSE) endpoint allows clients to subscribe to real-time 
updates for specific contexts running in the Node Server. Unlike WebSockets, which 
support two-way communication, SSE provides a one-way channel where the server continuously 
pushes updates to the client over a single long-lived HTTP connection. Subscriptions are 
automatically cleaned up after closing the connection.

The very first event received after opening the SSE stream is a `connect` event, which contains
a `connection_id`. This `connection_id` is required when making subscription requests.

#### SSE Subscription Handling:

To subscribe, the client must send a `POST` request to the `/sse/subscription` endpoint with 
one or more `contextIds`. 

#### SSE Unsubscription Handling:

To unsubscribe, the client must send a `POST` request to the `/sse/subscription` endpoint with 
one or more `contextIds`.

## Node Server Workflows

### Client Login Workflow

```mermaid
sequenceDiagram
    title Client Login Workflow

    participant User
    participant Admin Dashboard / Application
    participant Crypto Wallet
    participant Admin API
    participant Node

    User->>Admin Dashboard / Application: login
    Admin Dashboard / Application->>Admin API: API call to request-challenge endpoint
    Admin API-->>Admin Dashboard / Application: Return challenge object
    Admin Dashboard / Application->>Crypto Wallet: Request to sign received challenge
    Crypto Wallet-->>User: Request to sign challenge
    User->>Crypto Wallet: Sign challenge
    Crypto Wallet-->>Crypto Wallet: Sign challenge
    Crypto Wallet-->>Admin Dashboard / Application: Signed challenge
    Admin Dashboard / Application-->>Admin Dashboard / Application: Create auth headers
    Admin Dashboard / Application->>Admin API: call add-client-key endpoint with signed challenge + auth header
    Admin API->>Admin API: Verify auth headers
    Admin API->>Admin API: Verify signature
    Admin API->>Node: Check if any root key are stored
    alt No root keys are stored
        Admin API->>Node: Save root key
    else Root key exists
        Admin API->>Node: Save client key
    end
    Node->>Admin API: Key Stored response
    Admin API->>Admin Dashboard / Application: Login successful response
    Admin Dashboard / Application-->>Admin Dashboard / Application: Authorise user
```

### JSON rpc Workflow

```mermaid
sequenceDiagram
    title JSON rpc Workflow

    participant User
    participant Client
    participant JSON rpc
    participant Node

    User->>Client: Query/Mutate action
    Client->>JSON rpc: Request (query/mutate)
    JSON rpc-->>JSON rpc: Processes request
    JSON rpc->>Node: Request for JSON rpc action
    Node->>Node: Perform action (query/mutate)
    Node-->>JSON rpc: Response
    JSON rpc-->>Client: Response for request
```

### Websocket Workflow

```mermaid
sequenceDiagram
    title WebSocket Subscription Workflow

    participant Client
    participant WebSocket
    participant Node

    Client->>WebSocket: Subscribe/Unsubscribe request
    WebSocket->>Node: Handle subscribe/unsubscribe
    Node-->>WebSocket: Subscription response
    WebSocket-->>Client: Subscription messages (context application updates)
```

### SSE Workflow

```mermaid
sequenceDiagram
    title SSE Subscription Workflow

    participant Client
    participant SSE
    participant Node

    Client->>SSE: Open SSE connection
    SSE-->>Client: Send `connect` event with `connection_id`

    Client->>SSE: POST /sse/subscription (subscribe) 
    SSE->>Node: Register subscription for contextIds with connection_id
    Node-->>SSE: Stream events for contextIds
    SSE-->>Client: Push events as text/event-stream

    Client->>SSE: POST /sse/subscription (unsubscribe) 
    SSE->>Node: Removes subscription for contextIds
```

## Admin API endpoints

The Admin API endpoints are split into protected and unprotected routes, where
protected routes require authentication.

**Base path**: `/admin-api`

### Protected Routes

These routes require authentication. The authentication method depends on the auth mode:

- **Proxy mode (default)**: Uses auth headers generated with `createAuthHeader` from the `calimero sdk` library.
- **Embedded auth mode**: Uses JWT tokens in the `Authorization` header (e.g., `Authorization: Bearer <token>`).

**Auth Headers (Proxy Mode)**

Parts of the Auth Headers:

1.  `wallet_type`: Specifies the type of wallet used (e.g., NEAR).
2.  `signing_key`: Encoded public key used for signing the request.
3.  `signature`: Encoded signature generated from the payload hash.
4.  `challenge`: Encoded hash of the payload, serving as a challenge.
5.  `context_id`: Context identifier for additional request context. Optional
    for Admin Dashboard but mandatory for applications.

**1. Create Root Key**

- **Path**: `/root-key`
- **Method**: `POST`
- **Description**: Creates a new root key in the node.

**2. Install Application**

- **Path**: `/install-application`
- **Method**: `POST`
- **Description**: Installs a new application in the node.

**3. List Applications**

- **Path**: `/applications`
- **Method**: `GET`
- **Description**: Lists all installed applications in the node.

**4. Fetch DID**

- **Path**: `/did`
- **Method**: `GET`
- **Description**: Fetches the DID (Decentralized Identifier) of the node.

**5. Create Context**

- **Path**: `/contexts`
- **Method**: `POST`
- **Description**: Creates a new context.

**6. Delete Context**

- **Path**: `/contexts/:context_id`
- **Method**: `DELETE`
- **Description**: Deletes a specific context by ID.

**7. Get Context**

- **Path**: `/contexts/:context_id`
- **Method**: `GET`
- **Description**: Retrieves details of a specific context by ID.

**8. Get Context Users**

- **Path**: `/contexts/:context_id/users`
- **Method**: `GET`
- **Description**: Lists users associated with a specific context.

**9. Get Context Client Keys**

- **Path**: `/contexts/:context_id/client-keys`
- **Method**: `GET`
- **Description**: Lists client keys for a specific context.

**10. Get Context Storage**

- **Path**: `/contexts/:context_id/storage`
- **Method**: `GET`
- **Description**: Retrieves storage information for a specific context.

**11. List Contexts**

- **Path**: `/contexts`
- **Method**: `GET`
- **Description**: Lists all contexts.

**12. Delete Auth Keys**

- **Path**: `/identity/keys`
- **Method**: `DELETE`
- **Description**: Deletes all root and client keys.

### Unprotected Routes

These routes do not require authentication.

**1. Health Check**

- **Path**: `/admin-api/health`
- **Method**: `GET`
- **Description**: Checks the health of the API.

**2. Authentication Status**

- **Path**: `/admin-api/is-authed`
- **Method**: `GET`
- **Description**: Checks if the current request is authenticated.

**3. Certificate**

- **Path**: `/admin-api/certificate`
- **Method**: `GET`
- **Description**: Retrieves the node's certificate.

**4. Request Challenge**

- **Path**: `/request-challenge`
- **Method**: `POST`
- **Description**: Requests a challenge for authentication.

**5. Add Client Key**

- **Path**: `/add-client-key`
- **Method**: `POST`
- **Description**: Adds a new client key.

**6. Install Dev Application**

- **Path**: `/dev/install-application`
- **Method**: `POST`
- **Description**: Installs a development application.

**7. Manage Dev Contexts**

- **Path**: `/dev/contexts`
- Methods: `GET`, `POST`
- **Description**: Lists (`GET`) and creates (`POST`) development contexts.

**8. List Dev Applications**

- **Path**: `/dev/applications`
- **Method**: `GET`
- **Description**: Lists all development applications.

## JSON rpc endpoint

The JSON-rpc server endpoint is structured to handle various request types.

**Base path**: `/jsonrpc`

**Note**: When embedded auth mode is enabled, all JSON-RPC requests require authentication
via JWT tokens in the `Authorization` header (e.g., `Authorization: Bearer <token>`).

**1. Handle JSON-rpc Request**

- **Path**: `/jsonrpc`
- **Method**: `POST`
- **Description**: Handles incoming JSON-rpc requests, which can be `query` or
  `mutate` requests, processes them, and returns the appropriate response.

## Websocket endpoints

The WebSocket, accessible at /ws, allows clients to dynamically subscribe to and
unsubscribe from real-time updates about specific contexts within the Node
Server.

**Note**: When embedded auth mode is enabled, WebSocket connections require authentication
via JWT tokens in the `Authorization` header during the initial handshake.

**1. Handle WebSocket Request**

- **Path**: `/ws`
- **Method**: `GET`
- **Description**: Handles incoming WebSocket requests, which can be subscribe
  or unsubscribe requests, processes them, and returns the appropriate response.

## SSE endpoints

The SSE endpoint, accessible at `/sse`, allows clients to establish a real-time connection 
to receive updates for specific contexts within the Node Server.

**Note**: When embedded auth mode is enabled, SSE connections require authentication
via JWT tokens in the `Authorization` header when opening the connection.

**1. Establish SSE Connection**

- **Path**: `/sse`
- **Method**: `GET`
- **Description**: Opens a long-lived HTTP connection for receiving server-sent events.  
  The very first message received is a `connect` event containing a `connection_id`.  
  This `connection_id` must be used for subsequent subscription requests.
- **Example response (first event)**:
    ```text
    event: connect
    data: "connection_id"
    ```

**2. Subscribe to Context**

- **Path**: `/sse/subscription`
- **Method**: `POST`
- **Description**: Subscribes the active connection (identified by `connection_id`) to one or more `contextIds`.  
  After subscribing, updates for those contexts are streamed over the open SSE connection.
- **Example request**:
    ```http
    POST /sse/subscription
    Content-Type: application/json

    {
        "id": "connection_id received from the connect event from GET /sse",
        "method": "subscribe",
        "params": {
            "contextIds": ["context_1"]
        }
    }
    ```

**3. Unsubscribe from Context**

- **Path**: `/sse/subscription`
- **Method**: `POST`
- **Description**: To unsubscribe, clients send an `unsubscribe` request similar to subscription request
  with one or more `contextIds`.  
- **Example request**:
    ```http
    POST /sse/subscription
    Content-Type: application/json

    {
        "id": "connection_id received from the connect event from GET /sse",
        "method": "unsubscribe",
        "params": {
            "contextIds": ["context_1"]
        }
    }
    ```

## Examples

Examples of Node Server usage can be found within the
[Admin Dashboard](https://github.com/calimero-network/admin-dashboard) and the
[Only Peers example](https://github.com/calimero-network/only-peers-client)
application. All communication with the node is exposed through
[calimero sdk](https://github.com/calimero-network/core-js-sdk) library.