switchgear 0.1.18

High availability LNURL load balancer server for enterprise Bitcoin Lightning payment providers
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
# Switchgear

**By [Bitshock](https://bitshock.com) | [info@bitshock.com](mailto:info@bitshock.com)**

> "The impediment to action advances action. What stands in the way becomes the way." – Marcus Aurelius
 
Switchgear is a high availability LNURL balancer for enterprise Bitcoin Lightning payment providers. It is designed to scale massive multi-region Lightning Node fleets at five nines uptime.

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/switchgear.png)

## Features

Professional LNURL load balancing for your enterprise:

* CLN Lightning Node support with gRPC
* LND Lightning Node support with gRPC
* built on CloudFlare's [Pingora]https://github.com/cloudflare/pingora Load Balancer
* Three balancing algorithms: Round Robin, Random and [Consistent]https://en.wikipedia.org/wiki/Consistent_hashing (Ketama)
* Node health checks redirect invoice requests to healthy nodes
* Self-healing - if an unhealthy node transitions to healthy, it will automatically start taking invoice requests again
* Weighting can shift invoice requests to preferred nodes
* Liquidity biasing with inbound capacity checks will favor nodes most likely to accept payment
* Timed retries with exponential backoff for spurious node failures
* Partitions bind inbound invoice requests to subset of nodes
* Add nodes in seconds with REST Discovery API - new nodes will automatically start taking invoice requests
* Drop nodes in seconds with REST Discovery API - the balancer will safely direct invoices to remaining healthy nodes
* Publish new LNURLs in seconds with Offer API 
* `bech32` and QR code generation endpoints
* Sqlite, MySql and Postgres database support for both Discovery and Offer data stores
* Remote REST stores for both Discovery and Offer 

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/sequence-LNURL_Pay_Multi_Backend_Invoice_Generation.png)


## Status

Switchgear is in **ALPHA** status:

* Critical features are implemented and work as designed
* Integration tests are complete
* APIs may change without warning

See [ROADMAP.md](https://github.com/bitshock-src/switchgear/blob/HEAD/ROADMAP.md) for the Switchgear release roadmap.

## Why Bitcoin Lightning Payments Fail

### Single Points Of Failure

Failure with a standard LNURL service configuration is highly-probable: 

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/sequence_invoice_failure-LNURL_Pay___Invoice_Generation_Failure.png)


### No Liquidity Bias

Single LNURL + Lightning Node deployments make Liquidity Bias impossible. Liquidity bias deprioritizes nodes for invoice requests that have low inbound capacity, which make payment success unlikely.

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/sequence_payment_failure-LNURL_Pay___Payment_Failure__Inadequate_Inbound_Liquidity_.png)


### External Load Balancer Resource Exhaustion

Load balancing multiple LNURL instances with a single fixed Lightning Node will  exhaust resources:

* If a node fails, it makes the entire LNURL instance useless
* If a LNURL instance fails, it makes the Lightning Node unreachable
* Duplicating the entire LNURL instance just to attach it to a single Lightning Node raises cost, reducing feasibility of reaching scale
* High latency increases chances of catastrophic retry loop or flooding single node with failover requests

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/lnurl-external-balancer.png)

By contrast, every instance of Switchgear in a region can share the Lightning Node fleet, making scale feasible:

* A single Switchgear instance failure does not make a Lightning Node unreachable
* Any Lightning Node failure does not affect the uptime of a Switchgear instance
* Switchgear instances and the Lightning Node fleet size can scale independently, reducing cost
* Health checks are low latency, keeping Lightning Node status accurate
* Internal Backoff runs through entire Backend selection before returning 502 to external balancer, reducing latency and wallet retry cascade

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/switchgear-single-region.png)

The internal backoff can be entered under two conditions:

1. No healthy Lightning Nodes are available
2. A Lightning Node spurious failure or node failure between health checks

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/get_invoice_loop-Get_Invoice_Loop.png)

## Install

The Switchgear binary runs all services as well as the CLI admin interface.

### Host

```shell
cargo install switchgear-server
```

### Docker

The docker image is multi-platform for:

* linux/amd64
* linux/arm64

```shell
docker pull bitshock/switchgear
```

## Starting Switchgear Services

``` shell
swgr service --config {path/to/config/file} {service enabledment list}
```

The configuration file is in YAML format, and controls settings for all services.

The service enablement list can be any of:

* `lnurl` - the public LNURL service
* `discovery` - the admin discovery service
* `offer` - the offer admin service
* `all` - all services

If left empty, all services will be enabled (same as `all`).

### Docker

To run the Docker image:

```shell
docker run bitshock/switchgear
```

The image is configured with a default configuration file path of `/etc/swgr/config.yaml` . Mount a volume on top of `/etc/swgr` to provide your own configuration file.

To build the Docker image:

```shell
docker buildx build --platform linux/arm64,linux/amd64 -t swgr .
```

Set the build arg `WEBPKI_ROOTS=true` if you need Mozilla's web PKI roots bundle installed on the image:

```shell
docker buildx build --platform linux/arm64,linux/amd64 --build-arg WEBPKI_ROOTS=true -t swgr .
```

## Administration

Switchgear can be configured by both the REST API and the CLI.

### REST Administration

Administration Service endpoints:

```
https://{host}/discovery
https://{host}/offers
```

See the [Manage Lightning Node Backends with Discovery Service](#manage-lightning-node-backends-with-discovery-service) and [Manage LNURLs with Offer Service](#manage-lnurls-with-offer-service) sections for complete REST API.

### CLI Administration

```shell
# Manage Lightning Node Backends
swgr discovery
#  Manage LNURLs
swgr offer
```
See the [Manage Lightning Node Backends with Discovery Service](#manage-lightning-node-backends-with-discovery-service) and [Manage LNURLs with Offer Service](#manage-lnurls-with-offer-service) sections for complete CLI manual.

### Docker

To run the CLI administration from Docker:

```shell
docker run bitshock/switchgear {cli-options}
```

## Configuring Switchgear Services

All service configuration is controlled by a yaml file passed to the server at startup.

See [server/config](https://github.com/bitshock-src/switchgear/blob/HEAD/server/config) directory for more configuration examples.

Each service has a root entry the configuration file:

```yaml
# LNURL Service Configuration
lnurl-service:
  
# Discovery Service Configuration
discovery-service:

# Offer Service Configuration  
offer-service:
  
# Persistence Settings for Discovery and Offer data  
store:
```

See the service entries below for complete configuration manual.

#### Env Var Shell Expansion

Shell-style env var expansion is supported anywhere in the yaml configuration file.

Example config.yaml:

```yaml
lnurl-service:
  address: "${MY_LNURL_SERVICE_ADDRESS:-127.0.0.1:8080}"
```

Run with:

```shell
MY_LNURL_SERVICE_ADDRESS=192.168.1.100:8080 swgr service --config ./config.yaml
```

The configuration would be parsed as:

```yaml
lnurl-service:
  address: "192.168.1.100:8080"
```

If the env var is unset: 

```shell
swgr service --config ./config.yaml
```

The configuration would be parsed as:

```yaml
lnurl-service:
  address: "127.0.0.1:8080"
```

## Liquidity Bias

Optional Liquidity Bias will prioritize nodes with inbound capacity at the moment of selection. If no nodes are in range of the capacity bias, selection will fall back to standard weight selection.

#### Negative Capacity Bias example (restrictive):

```yaml
lnurl-service:
  selection-capacity-bias: -0.2
```

Invoice request amount must be less than 20% of total inbound capacity of the node to be favored over other nodes that are not.

#### Positive Capacity Bias example (permissive):

```yaml
lnurl-service:
  selection-capacity-bias: 0.1
```

Invoice request amount may be over inbound capacity by up to 10% of the node and still be favored over other nodes that are not.

#### Inbound Capacity Measurement

Capacity is measured in the same cycle as the Lightning Node health check. It is the sum of inbound capacity for all active channels on the node.

## Partitioning

An organization may have a global Offer database. A Switchgear instance may be configured to serve a portion of that database, using partitions. Furthermore, every Lightning Node is configured in Discovery to be bound to one or more partitions, insuring payments only land on nodes they belong to.

Each Switchgear instance is configured for the partitions it will serve.

Example:

```yaml
lnurl-service:
  partitions: ["us", "it", "cr"]
```

Only Offers created in "us", "it" or "cr" partitions will be available on the Switchgear instance, even if the Offer exists the database: 

```
https://example.com/offers/us/{id} - 200 success
https://example.com/offers/it/{id} - 200 success 
https://example.com/offers/cr/{id} - 200 success
https://example.com/offers/ca/{id} - 404 not found
```

Each partition must have Discovery Backends configured for the invoice request to succeed. A single backend can serve multiple partitions.

See the [Manage Lightning Node Backends with Discovery Service](#manage-lightning-node-backends-with-discovery-service) and [Manage LNURLs with Offer Service](#manage-lnurls-with-offer-service) sections for creating Discovery Backends and Offers with partitions.

## Balancing Switchgear

Switchgear itself can be balanced. Balancing multiple switchgear instances within a region:

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/switchgear-single-region.png)

For multi-region Global Load Balancer deployment, use the full health check to signal to the downstream balancer that a Switchgear instance has no healthy Lightning Nodes:

```
https://{host}/health/full
```

The full health check will return 500 if no Lightning Nodes are available.

The regional balancer will forward the failing health status to the global balancer, which will send invoice requests to an alternate region:

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/switchgear-multi-region.png)

Switchgear partitions have predictable URLs. Use partitions and a global Application Load Balancer to map invoice requests to alternate regions that have Switchgear instances configured for the requested partition.

## LNURL Service

The OpenAPI LNURL Service specification: [doc/lnurl-service-openapi.yaml](https://github.com/bitshock-src/switchgear/blob/HEAD/doc/lnurl-service-openapi.yaml).

The LNURL Service is public facing, and implements the [LNURL LUD-06 specification.](https://github.com/lnurl/luds/blob/luds/06.md)

See the [Manage LNURLs with Offer Service](#manage-lnurls-with-offer-service) section for complete service manual.

All Switchgear LNURLs are formatted as:

```
https://{host}/offers/{partition}/{id}
```

Where:

* `partition` - the Offer partition
* `id` - the Offer id (Uuid)

The returned callback is always the LNURL, with the postfix `/invoice` :

```
https://{host}/offers/{partition}/{id}/invoice
```

The bech32 and QR variants are available with:

```
https://{host}/offers/{partition}/{id}/bech32
```

And:

```
https://{host}/offers/{partition}/{id}/bech32/qr
```

The QR image is in PNG format.

### LNURL Service Configuration

See [server/config](https://github.com/bitshock-src/switchgear/blob/HEAD/server/config) directory for more configuration examples.

```yaml
lnurl-service:
  # List of partitions this service will handle
  # Partitions allow you to segment different Lightning node groups
  partitions: ["default"]
  
  # Network address and port for the LNURL service to bind to
  address: "127.0.0.1:8080"
  
  # Frequency in seconds for health checking Lightning node backends (float)
  health-check-frequency-secs: 1.0
  
  # Whether to perform health checks in parallel across all backends
  parallel-health-check: true
  
  # Number of consecutive successful health checks needed to mark a backend as healthy
  health-check-consecutive-success-to-healthy: 1
  
  # Number of consecutive failed health checks needed to mark a backend as unhealthy
  health-check-consecutive-failure-to-unhealthy: 1
  
  # Frequency in seconds for updating backend node information (float)
  backend-update-frequency-secs: 1.0
  
  # Invoice expiry time in seconds (integer)
  invoice-expiry-secs: 180
  
  # Timeout in seconds for Lightning node client connections (float)
  ln-client-timeout-secs: 2.0

  # Optional trusted roots pem bundle for all LN clients
  ln-trusted-roots: "/etc/ssl/certs/ln-ca.pem"
  
  # List of allowed host headers for incoming requests
  # Used for safely generating callback/invoice URLs.
  allowed-hosts: ["lnurl.example.com"]
  
  # Backoff configuration for retrying failed operations.
  # Backoff is used when Lightning Node invoice request fails.
  backoff:
    # Type of backoff strategy: "stop" or "exponential"
    type: "exponential"
    # Optional: Initial interval in seconds for exponential backoff (float)
    initial-interval-secs: 1.0
    # Optional: Randomization factor (0.0 to 1.0)
    randomization-factor: 0.5
    # Optional: Multiplier for each retry attempt
    multiplier: 2.0
    # Optional: Maximum interval between retries in seconds (float)
    max-interval-secs: 60.0
    # Optional: Maximum total elapsed time before giving up in seconds (float)
    max-elapsed-time-secs: 300.0
  
  # Backend selection strategy for load balancing
  # Options: "round-robin", "random", or "consistent"
  backend-selection: "round-robin"
  # For consistent hashing, specify max iterations (only used with "consistent")
  # backend-selection:
  #   type: "consistent"
  #   max-iterations: 10000
    
  # Optional: Bias factor for capacity-influenced selection
  # Negative values are restrictive:
  # prefer nodes with capacity higher than requested amount
  # Positive values are lenient:
  # refer nodes with capacity less than requested amount  
  selection-capacity-bias: -0.2

  # Optional: Allow &comment query param in LNURL invoice request, sized in char len
  # Used for Consistent backend selection
  comment_allowed: 64,

  # Optional: TLS configuration for HTTPS support
  tls:
    # Path to TLS certificate file
    cert-path: "/etc/ssl/certs/lnurl-cert.pem"
    # Path to TLS private key file
    key-path: "/etc/ssl/certs/lnurl-key.pem"
    
  # QR module width x height
  bech32-qr-scale: 8
  # QR light gray level
  bech32-qr-light: 255
  # QR dark gray level
  bech32-qr-dark: 0
```

### Consistent Backend-Selection

Consistent uses the optional LNURL `comment` query parameter as a hash key, which guarantees the same node will always receive invoice requests for that key. The balancer will move on to the next closest key match if the node becomes unavailable. This is a specific use-case that provides optimized HTLC settlement between cooperating peers for high-frequency transactions.

## Discovery Service

The OpenAPI Discovery Service specification: [doc/discovery-service-openapi.yaml](https://github.com/bitshock-src/switchgear/blob/HEAD/doc/discovery-service-openapi.yaml).

The Discovery Service is an administrative service used to manage connections to individual Lightning Nodes.

The service is isolated from the LNURL Service and can be configured to run on any port. The service supports TLS. Access is protected by a bearer token. Do not run this service without TLS enabled if it is exposed to the public internet.

See the [Manage Lightning Node Backends with Discovery Service](#manage-lightning-node-backends-with-discovery-service) section for complete service manual.

### Discovery Service Configuration

See [server/config](https://github.com/bitshock-src/switchgear/blob/HEAD/server/config) directory for more configuration examples.

```yaml
discovery-service:
  # Network address and port for the Discovery service to bind to
  address: "127.0.0.1:8081"
  
  # Path to the authentication authority certificate/key file
  # This file contains the public key used to verify API access
  auth-authority: "/etc/ssl/certs/discovery-auth-authority.pem"
  
  # Optional: TLS configuration for HTTPS support
  tls:
    # Path to TLS certificate file
    cert-path: "/etc/ssl/certs/discovery-cert.pem"
    # Path to TLS private key file
    key-path: "/etc/ssl/certs/discovery-key.pem"
```

#### Authentication Setup

Generate key pairs and tokens for Discovery service authentication:

```shell
# Generate a new key pair for token signing
swgr discovery token key --public discovery-public.pem --private discovery-private.pem

# The public key (discovery-public.pem) should be used as the auth-authority in the configuration
# The private key (discovery-private.pem) is used to mint authentication tokens

# Create a token (default 3600 seconds)
swgr discovery token mint --key discovery-private.pem --output discovery.token
```

## Offer Service

The OpenAPI Offer Service specification: [doc/offer-service-openapi.yaml](https://github.com/bitshock-src/switchgear/blob/HEAD/doc/offer-service-openapi.yaml).

The Offer Service is an administrative service used to manage Offers, which are used to generate LNURLs.

The service is isolated from the LNURL Service and can be configured to run on any port. The service supports TLS. Access is protected by a bearer token. Do not run this service without TLS enabled if it is exposed to the public internet.

See the [Manage LNURLs with Offer Service](#manage-lnurls-with-offer-service) section for complete service manual.

### Offer Service Configuration

See [server/config](https://github.com/bitshock-src/switchgear/blob/HEAD/server/config) directory for more configuration examples.

```yaml
offer-service:
  # Network address and port for the Offers service to bind to
  address: "127.0.0.1:8082"
  
  # Path to the authentication authority certificate/key file
  # This file contains the public key used to verify API access
  auth-authority: "/etc/ssl/certs/offer-auth-authority.pem"
  
  # Optional: TLS configuration for HTTPS support
  tls:
    # Path to TLS certificate file
    cert-path: "/etc/ssl/certs/offer-cert.pem"
    # Path to TLS private key file
    key-path: "/etc/ssl/certs/offer-key.pem"

  # max page size for get all queries
  max-page-size: 100
```

#### Authentication Setup

Generate key pairs and tokens for Offer service authentication:

```shell
# Generate a new key pair for token signing
swgr offer token key --public offer-public.pem --private offer-private.pem

# The public key (offer-public.pem) should be used as the auth-authority in the configuration
# The private key (offer-private.pem) is used to mint authentication tokens

# Create a token (default 3600 seconds)
swgr offer token mint --key offer-private.pem --output offer.token
```

## Persistence

Both Discovery and Offer services support multiple storage backends. Configure persistence in the `store` section of your configuration file.

### Common Storage Types

Both Discovery and Offer stores support these storage backends:

#### Database Storage (SQLite/MySQL/PostgreSQL)

```yaml
store:
  discover:  # or 'offer'
    type: "database"
    # Database connection URL (SQLite/MySQL/PostgreSQL)
    database-url: "connection-url"
    # Maximum number of concurrent database connections
    max-connections: 5
```

For `database-url` formats, see [Database Connection URLs](#database-connection-urls).

#### HTTP Storage (Remote Service)

Both Discovery and Offer can use a remote http store, making custom integrations straightforward. The store clients connect to the same REST API used for remote administration, making it possible for Switchgear to run headless as well, serving only as a database for other Switchgear instances.

```yaml
store:
  discover:  # or 'offer'
    type: "http"
    # Base URL of the remote service
    base-url: "https://service.example.com"
    # Timeout in seconds for establishing connection
    connect-timeout-secs: 2.0
    # Total timeout in seconds for complete request/response
    total-timeout-secs: 5.0
    # Optional pem bundle of trusted CA certificate paths for TLS verification
    trusted-roots: "/etc/ssl/certs/ca.pem"
    # Path to bearer token file for authentication
    authorization: "/etc/ssl/certs/auth.token"
```

#### In-memory Storage

Volatile storage, data is lost on restart:

```yaml
store:
  discover:  # or 'offer'
    type: "memory"
```

### Configuration Examples

#### Using Same Database for Both Stores

```yaml
store:
  # Discovery backend storage
  discover:
    type: "database"
    database-url: "postgres://user:password@localhost:5432/switchgear"
    max-connections: 5
  
  # Offer storage (sharing same database)
  offer:
    type: "database"
    database-url: "postgres://user:password@localhost:5432/switchgear"
    max-connections: 10
```

#### Mixed Storage Types

```yaml
store:
  # Memory storage for Discovery
  discover:
    type: "memory"
  
  # Database storage for Offers
  offer:
    type: "database"
    database-url: "sqlite:///var/lib/switchgear/offers.db?mode=rwc"
    max-connections: 5
```

#### Remote Service Configuration

```yaml
store:
  # Connect to remote Discovery service
  discover:
    type: "http"
    base-url: "https://discovery.internal:8081"
    connect-timeout-secs: 2.0
    total-timeout-secs: 5.0
    trusted-roots: "/etc/ssl/certs/internal-ca.pem"
    authorization: "/etc/ssl/certs/discovery.token"
  
  # Local database for Offers
  offer:
    type: "database"
    database-url: "sqlite:///data/offers.db?mode=rwc"
    max-connections: 10
```

### Database Connection URLs

Both Discovery and Offer data stores have a `database-url` field to configure the database.

### Sqlite

```
sqlite:///path/to/file.db?{options}
```

See [https://www.sqlite.org/uri.html](https://www.sqlite.org/uri.html) for all connection URL options.

### MySQL

```
mysql://[host][/database][?properties]
```

Properties:

| Parameter                  | Default     | Description                                                                                                       |
|----------------------------|-------------|-------------------------------------------------------------------------------------------------------------------|
| `ssl-mode`                 | `PREFERRED` | Determines whether or with what priority a secure SSL TCP/IP connection will be negotiated. See [`MySqlSslMode`]. |
| `ssl-ca`                   | `None`      | Sets the name of a file containing a list of trusted SSL Certificate Authorities.                                 |
| `statement-cache-capacity` | `100`       | The maximum number of prepared statements stored in the cache. Set to `0` to disable.                             |
| `socket`                   | `None`      | Path to the unix domain socket, which will be used instead of TCP if set.                                         |

### Postgres

```text
postgresql://[user[:password]@][host][:port][/dbname][?param1=value1&...]
```

See [https://www.postgresql.org/docs/current/libpq-connect.html](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING) for all connection URL options.

## Manage Lightning Node Backends With Discovery Service

The Discovery Service manages Lightning Node backends that connect to Switchgear for invoice requests. The service supports dynamic registration, updates, enablement and removal of CLN and LND nodes.

The Discovery Service can be administered with both REST and the CLI.

To get started quickly, use the CLI to write a new JSON data model template to a file:

```shell
# Generate a template backend configuration
swgr discovery new cln-grpc --output cln-backend.json
swgr discovery new lnd-grpc --output lnd-backend.json
````

### REST API

The Discovery Service provides a REST API for backend management. All endpoints except `/health` require bearer token authentication.

#### Authentication

First, generate a token:
```shell
# Create a token (expires in 3600 seconds)
# Note: Requires private key from Authentication Setup (see Discovery Service Configuration)
swgr discovery token mint --key discovery-private.pem --expires 3600 --output discovery.token

# Set authorization header for curl commands
export AUTH_TOKEN=$(cat discovery.token)
```

#### Register A New Backend

```shell
# Register a CLN node
curl -X POST http://localhost:3001/discovery \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "partitions": ["default"],
    "address": {
      "publicKey": "0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"
    },
    "name": "CLN Node 1",
    "weight": 100,
    "enabled": true,
    "implementation": {
      "type": "clnGrpc",
      "url": "https://192.168.1.100:9736",
      "domain": "cln-node.local",
      "auth": {
        "type": "path",
        "caCertPath": "/path/to/ca.pem",
        "clientCertPath": "/path/to/client.pem",
        "clientKeyPath": "/path/to/client-key.pem"
      }
    }
  }'

# Register an LND node
curl -X POST http://localhost:3001/discovery \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "partitions": ["default", "us", "eu"],
    "address": {
      "url": "https://lnd-node.example.com"
    },
    "name": "LND Node 1",
    "weight": 50,
    "enabled": true,
    "implementation": {
      "type": "lndGrpc",
      "url": "https://192.168.1.101:10009",
      "domain": "lnd-node.local",
      "auth": {
        "type": "path",
        "tlsCertPath": "/path/to/tls.cert",
        "macaroonPath": "/path/to/admin.macaroon"
      },
      "ampInvoice": false
    }
  }'
```

#### List All Backends

```shell
curl -X GET http://localhost:3001/discovery \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Get A Specific Backend

```shell
# By public key
curl -X GET "http://localhost:3001/discovery/pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" \
  -H "Authorization: Bearer $AUTH_TOKEN"

# By URL (base64 encoded)
curl -X GET "http://localhost:3001/discovery/url/aHR0cHM6Ly9sbmQtbm9kZS5leGFtcGxlLmNvbS8" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Update A Backend

```shell
curl -X PUT "http://localhost:3001/discovery/pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "partitions": ["default", "us"],
    "weight": 200,
    "enabled": false,
    "implementation": {
      "type": "clnGrpc",
      "url": "https://192.168.1.100:9736",
      "domain": "cln-node.local",
      "auth": {
        "type": "path",
        "caCertPath": "/path/to/ca.pem",
        "clientCertPath": "/path/to/client.pem",
        "clientKeyPath": "/path/to/client-key.pem"
      }
    }
  }'
```

#### Delete A Backend

```shell
curl -X DELETE "http://localhost:3001/discovery/pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Health Check

```shell
# No authentication required
curl http://localhost:3001/health
```

### CLI

The `swgr discovery` command provides the same functionality as the REST interface for remote management of Lightning Node backends.

#### Token Management

```shell
# Create a token (default 3600 seconds)
# Note: Requires a private key generated during Authentication Setup (see Discovery Service Configuration)
swgr discovery token mint --key discovery-private.pem --output discovery.token

# Create a token with custom expiration (86400 seconds = 24 hours)
swgr discovery token mint --key discovery-private.pem --expires 86400 --output discovery-24h.token

# Verify a token
swgr discovery token verify --public discovery-public.pem --token discovery.token
```

#### Backend Management

```shell
# Generate a template backend configuration
swgr discovery new cln-grpc --output cln-backend.json
swgr discovery new lnd-grpc --output lnd-backend.json

# Set connection parameters (via environment or flags)
export DISCOVERY_STORE_HTTP_BASE_URL="https://discovery.example.com"
export DISCOVERY_STORE_HTTP_AUTHORIZATION="/path/to/discovery.token"
export DISCOVERY_STORE_HTTP_TRUSTED_ROOTS="/path/to/ca.pem"

# List all backends (simple table format)
swgr discovery ls


# Get backend details (JSON output)
swgr discovery get pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 --output backend-details.json

# Get all backends (JSON output)
swgr discovery get

# Register a new backend from JSON file
swgr discovery post --input cln-backend.json

# Update an existing backend
swgr discovery put pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 --input updated-backend.json

# Patch an existing backend
swgr discovery patch pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 --input backend-patch.json

# Enable an existing backend
swgr discovery enable pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 

# Disable an existing backend
swgr discovery disable pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 

# Delete a backend
swgr discovery delete pk/0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
```

### Discovery Data Model

Discovery OpenAPI schema: [doc/discovery-service-openapi.yaml](https://github.com/bitshock-src/switchgear/blob/HEAD/doc/discovery-service-openapi.yaml).

Example CLN backend configuration:
```json
{
  "partitions": ["default"],
  "address": {
    "publicKey": "0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"
  },
  "name": "CLN Node 1",
  "weight": 1,
  "enabled": true,
  "implementation": {
    "type": "clnGrpc",
    "url": "https://127.0.0.1:9736",
    "domain": "localhost",
    "auth": {
      "type": "path",
      "caCertPath": "/path/to/ca.pem",
      "clientCertPath": "/path/to/client.pem",
      "clientKeyPath": "/path/to/client-key.pem"
    }
  }
}
```

Example LND backend configuration:
```json
{
  "partitions": ["default", "us", "eu"],
  "address": {
    "publicKey": "0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"
  },
  "name": "LND Node 1",
  "weight": 1,
  "enabled": true,
  "implementation": {
    "type": "lndGrpc",
    "url": "https://127.0.0.1:10009",
    "domain": "localhost",
    "auth": {
      "type": "path",
      "tlsCertPath": "/path/to/tls.cert",
      "macaroonPath": "/path/to/admin.macaroon"
    },
    "ampInvoice": false
  }
}
```



## Manage LNURLs With Offer Service

The Offer service manages Lightning payment offers and their metadata. It provides storage and retrieval of LNURL Pay offers with configurable payment limits and display information.

The Offer Service can be administered with both REST and the CLI.

To get started quickly, use the CLI to write a new JSON data model template to a file:

```shell
# Generate a template offer configuration
swgr offer new --output offer-template.json

# Generate a template metadata configuration
swgr offer metadata new --output metadata-template.json
````

### LNURL

An Offer has two fields that combine to create a unique identifier:

* `partition`
* `id` (Uuid)

Both fields are used to make the final public LNURL:

```
https://{host}/offers/{partition}/{id}
```


### REST API

The Offer service provides a REST API for offer and metadata management. All endpoints except `/health` require bearer token authentication.

#### Authentication

First, generate a token:
```shell
# Create a bearer token (expires in 3600 seconds)
# Note: Requires private key from Authentication Setup (see Offer Service Configuration)
swgr offer token mint --key offer-private.pem --expires 3600 --output offer.token

# Set authorization header for curl commands
export AUTH_TOKEN=$(cat offer.token)
```

#### Create A New Offer

```shell
curl -X POST http://localhost:3002/offers \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "partition": "default",
    "id": "6a38ebdd-83ef-4b94-b843-3b18cd90a833",
    "maxSendable": 1000000,
    "minSendable": 1000,
    "metadataId": "88deff7e-ca45-4144-8fca-286a5a18fb1a",
    "timestamp": "2024-01-01T00:00:00Z",
    "expires": "2024-12-31T23:59:59Z"
  }'
```

#### List All Offers In A Partition

```shell
curl -X GET http://localhost:3002/offers/default \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Get A Specific Offer

```shell
curl -X GET "http://localhost:3002/offers/default/6a38ebdd-83ef-4b94-b843-3b18cd90a833" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Update An Offer

```shell
curl -X PUT "http://localhost:3002/offers/default/6a38ebdd-83ef-4b94-b843-3b18cd90a833" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "maxSendable": 2000000,
    "minSendable": 1000,
    "metadataId": "88deff7e-ca45-4144-8fca-286a5a18fb1a",
    "timestamp": "2024-01-01T00:00:00Z",
    "expires": "2024-12-31T23:59:59Z"
  }'
```

#### Delete An Offer

```shell
curl -X DELETE "http://localhost:3002/offers/default/6a38ebdd-83ef-4b94-b843-3b18cd90a833" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Create Metadata

```shell
curl -X POST http://localhost:3002/metadata \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "88deff7e-ca45-4144-8fca-286a5a18fb1a",
    "partition": "default",
    "text": "Lightning Payment",
    "longText": "Pay for premium services with Lightning Network",
    "image": {
      "png": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="
    },
    "identifier": {
      "email": "payments@example.com"
    }
  }'
```

#### List All Metadata In A Partition

```shell
curl -X GET http://localhost:3002/metadata/default \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Get Specific Metadata

```shell
curl -X GET "http://localhost:3002/metadata/default/88deff7e-ca45-4144-8fca-286a5a18fb1a" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Update Metadata

```shell
curl -X PUT "http://localhost:3002/metadata/default/88deff7e-ca45-4144-8fca-286a5a18fb1a" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Updated Lightning Payment",
    "longText": "Pay for premium services with Lightning Network - Updated",
    "identifier": {
      "email": "billing@example.com"
    }
  }'
```

#### Delete Metadata

```shell
curl -X DELETE "http://localhost:3002/metadata/default/88deff7e-ca45-4144-8fca-286a5a18fb1a" \
  -H "Authorization: Bearer $AUTH_TOKEN"
```

#### Health Check

```shell
# No authentication required
curl http://localhost:3002/health
```

### CLI

The `swgr offer` command provides the same functionality as the REST interface for remote management of Lightning payment offers and metadata.

#### Token Management

```shell
# Create a token (default 3600 seconds)
# Note: Requires a private key generated during Authentication Setup (see Offer Service Configuration)
swgr offer token mint --key offer-private.pem --output offer.token

# Create a token with custom expiration (86400 seconds = 24 hours)
swgr offer token mint --key offer-private.pem --expires 86400 --output offer-24h.token

# Verify a token
swgr offer token verify --public offer-public.pem --token offer.token
```

#### Offer Management

```shell
# Generate a template offer configuration
swgr offer new --output offer-template.json

# Set connection parameters (via environment or flags)
export OFFER_STORE_HTTP_BASE_URL="https://offer.example.com"
export OFFER_STORE_HTTP_AUTHORIZATION="/path/to/offer.token"
export OFFER_STORE_HTTP_TRUSTED_ROOTS="/path/to/ca.pem"

# Get offer details (JSON output)
swgr offer get default 6a38ebdd-83ef-4b94-b843-3b18cd90a833 --output offer-details.json

# Get all offers in partition (JSON output)
swgr offer get default

# Create a new offer from JSON file
swgr offer post --input offer.json

# Update an existing offer
swgr offer put default 6a38ebdd-83ef-4b94-b843-3b18cd90a833 --input updated-offer.json

# Delete an offer
swgr offer delete default 6a38ebdd-83ef-4b94-b843-3b18cd90a833
```

#### Metadata Management

```shell
# Generate a template metadata configuration
swgr offer metadata new --output metadata-template.json

# Get metadata details (JSON output)
swgr offer metadata get default 88deff7e-ca45-4144-8fca-286a5a18fb1a --output metadata-details.json

# Get all metadata in partition (JSON output)
swgr offer metadata get default

# Create new metadata from JSON file
swgr offer metadata post --input metadata.json

# Update existing metadata
swgr offer metadata put default 88deff7e-ca45-4144-8fca-286a5a18fb1a --input updated-metadata.json

# Delete metadata
swgr offer metadata delete default 88deff7e-ca45-4144-8fca-286a5a18fb1a
```

### Offer Data Model

Offer OpenAPI schema: [doc/offer-service-openapi.yaml](https://github.com/bitshock-src/switchgear/blob/HEAD/doc/offer-service-openapi.yaml).

Example offer configuration:
```json
{
  "partition": "default",
  "id": "6a38ebdd-83ef-4b94-b843-3b18cd90a833",
  "maxSendable": 1000000,
  "minSendable": 1000000,
  "metadataId": "88deff7e-ca45-4144-8fca-286a5a18fb1a",
  "timestamp": "1970-01-01T00:00:00Z",
  "expires": null
}
```

Example metadata configuration:
```json
{
  "id": "88deff7e-ca45-4144-8fca-286a5a18fb1a",
  "partition": "default",
  "text": "mandatory offer text",
  "longText": "optional long offer text",
  "image": {
    "png": "base64_encoded_png_data"
  },
  "identifier": {
    "email": "optional@email.com"
  }
}
```

#### Image Support

Metadata can include images in PNG or JPEG format, base64 encoded:
- PNG: `{"png": "base64_encoded_data"}`
- JPEG: `{"jpeg": "base64_encoded_data"}`

#### Identifier Types

Metadata identifiers can be:
- Email: `{"email": "contact@example.com"}`
- Text: `{"text": "contact@example.com"}`

## SDK 

### Service

The [switchgear-service](https://github.com/bitshock-src/switchgear/blob/HEAD/service) crate defines all services and their trait dependencies. See the `api` module for trait definitions and data models: [service/src/api](https://github.com/bitshock-src/switchgear/blob/HEAD/service/src/api)

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/service_traits_component_diagram-Service_Layer_Trait_Relationships.png)


### Pingora

`PingoraLnBalancer` is the default `LnBalancer` implementation. The [switchgear-pingora](https://github.com/bitshock-src/switchgear/blob/HEAD/pingora) crate holds the complete implementation, plus trait definitions it uses for itself.

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/pingora_traits_component_diagram-PingoraLnBalancer_Trait_Dependencies.png)


### Components

The `components` module in [switchgear-service](https://github.com/bitshock-src/switchgear/blob/HEAD/service/src/components) is a collection self-defined traits and implementations useful for implementing a complete `LnBalancer`. The module also holds different implementations of `DiscoveryBackendStore`, `OfferStore` and `OfferMetadataStore`.

#### Service Components 

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/service_components_traits_diagram-Service_Components_Trait_Dependencies.png)

#### Data Store Implementations

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/service_discovery_traits_diagram-Discovery_Components_Trait_Dependencies.png)

![image](https://raw.githubusercontent.com/bitshock-src/switchgear/main/doc/service_offer_traits_diagram-Offer_Components_Trait_Dependencies.png)