2.2 Electrum Server: Fulcrum

Fulcrum is a fast & nimble SPV server for Bitcoin created by Calin Culianu. It can be used as an alternative to Electrs because of its performance, as we can see in Craig Raw's comparison of servers.

Requirements

• Bitcoin Core

• ~ 130GB of free storage for the database

Fulcrum is a replacement for Electrs, these two services cannot be run at the same time (due to the same standard ports used), remember to stop Electrs doing sudo systemctl stop electrs.

Introduction

Bitcoin with hardware wallets

The best way to safely keep your bitcoin (meaning the best combination of security and usability) is to use a hardware wallet (like BitBox, Coldcard, Ledger, or Trezor) in combination with your own Bitcoin node. This gives you security, privacy and eliminates the need to trust a third party to verify transactions.

Bitcoin Core on the MiniBolt itself is not meant to hold funds.

One possibility to use Bitcoin Core with your Bitcoin wallets is to use an Electrum server as middleware. It imports data from Bitcoin Core and provides it to software wallets supporting the Electrum protocol. Desktop wallets like Sparrow, the BitBoxApp, Electrum, or Specter Desktop that support hardware wallets can then be used with your own sovereign Bitcoin node.

Preparations

Make sure that you have reduced the database cache of Bitcoin Core

Install dependencies

• With user admin, update and upgrade your OS

sudo apt update && sudo apt full-upgrade

• Make sure that all necessary software packages are installed

sudo apt install libssl-dev

Configure Firewall

• Configure the firewall to allow incoming requests

sudo ufw allow 50001/tcp comment 'allow Fulcrum TCP from anywhere'

sudo ufw allow 50002/tcp comment 'allow Fulcrum SSL from anywhere'

Configure Bitcoin Core

We need to set up settings in the Bitcoin Core configuration file - add new lines if they are not present

• Edit bitcoin.conf file

sudo nano /data/bitcoin/bitcoin.conf

• Add the following line to the "# Connections" section. Save and exit

zmqpubhashblock=tcp://127.0.0.1:8433

• Restart Bitcoin Core to apply changes

sudo systemctl restart bitcoind

• Check Bitcoin Core is enabled zmqpubhashblock on the 8433 port

sudo ss -tulpn | grep LISTEN | grep bitcoind | grep 8433

Expected output:

> tcp   LISTEN 0      100        127.0.0.1:       0.0.0.0:*    users:(("bitcoind",pid=773834,fd=18))

Installation

Download binaries

We have our Bitcoin Core configuration file set up and can now move on to the next part of the Fulcrum installation.

• Login as admin user and change to a temporary directory which is cleared on reboot

cd /tmp

• Set a temporary version environment variable to the installation

VERSION=1.11.0

• Download the application, checksums, and signature

wget https://github.com/cculianu/Fulcrum/releases/download/v$VERSION/Fulcrum-$VERSION-x86_64-linux.tar.gz

wget https://github.com/cculianu/Fulcrum/releases/download/v$VERSION/Fulcrum-$VERSION-shasums.txt.asc

wget https://github.com/cculianu/Fulcrum/releases/download/v$VERSION/Fulcrum-$VERSION-shasums.txt

Signature check

• Get the public key from the Fulcrum developer

curl https://raw.githubusercontent.com/Electron-Cash/keys-n-hashes/master/pubkeys/calinkey.txt | gpg --import

Expected output:

[...]
>     % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                 Dload  Upload   Total   Spent    Left  Speed
> 100  2328  100  2328    0     0  16043      0 --:--:-- --:--:-- --:--:-- 15945
> gpg: key 21810A542031C02C: "Calin Culianu (NilacTheGrim) <[email protected]>" 
> gpg: Total number processed: 1
> gpg:              unchanged: 1
[...]

• Verify the signature of the text file containing the checksums for the application

gpg --verify Fulcrum-$VERSION-shasums.txt.asc

Expected output:

[...]
> gpg:  from "Calin Culianu (NilacTheGrim) <[email protected]>" [unknown]
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the owner.
> Primary key fingerprint: D465 135F 97D0 047E 18E9  9DC3 2181 0A54 2031 C02C
[...]

Checksum check

• Verify the signed checksum against the actual checksum of your download

grep 'x86_64-linux.tar.gz' Fulcrum-$VERSION-shasums.txt | sha256sum --check

Example of expected output:

> 

• Extract

tar -xvf Fulcrum-$VERSION-x86_64-linux.tar.gz

Binaries installation

• Install it

sudo install -m 0755 -o root -g root -t /usr/local/bin Fulcrum-$VERSION-x86_64-linux/Fulcrum Fulcrum-$VERSION-x86_64-linux/FulcrumAdmin

• Check the correct installation requesting the version

Fulcrum --version

Example of expected output:

> Fulcrum 1.9.1 (Release 713d2d7)
> compiled: gcc 8.4.0
[...]

• (Optional) Delete installation files of the tmp folder

sudo rm -r Fulcrum-$VERSION-x86_64-linux Fulcrum-$VERSION-x86_64-linux.tar.gz Fulcrum-$VERSION-shasums.txt Fulcrum-$VERSION-shasums.txt.asc

If you come to update, this is the final step, go back to the Upgrade section to continue

Create the fulcrum user & group

• Create the fulcrum user and group

sudo adduser --disabled-password --gecos "" fulcrum

• Add fulcrumuser to the "bitcoin" group, allowing to the fulcrumuser read the bitcoind .cookie file

sudo adduser fulcrum bitcoin

Create data folder

• Create the fulcrum data folder

sudo mkdir -p /data/fulcrum/fulcrum_db

• Assign as the owner to the fulcrum user

sudo chown -R fulcrum:fulcrum /data/fulcrum/

• Change to the fulcrum user

sudo su - fulcrum

• Create a symlink from /home/fulcrum/.fulcrum folder to the /data/fulcrum folder

ln -s /data/fulcrum /home/fulcrum/.fulcrum

• Check symbolic link has been created correctly

ls -la

Expected output:

total 24
drwxr-x--- 2 fulcrum fulcrum 4096 Jul 15 07:59 .
drwxr-xr-x 6 root    root    4096 Jul 15 07:56 ..
-rw------- 1 fulcrum fulcrum   24 Jul 15 07:59 .bash_history
-rw-r--r-- 1 fulcrum fulcrum  220 Jul 15 07:56 .bash_logout
-rw-r--r-- 1 fulcrum fulcrum 3771 Jul 15 07:56 .bashrc
lrwxrwxrwx 1 fulcrum fulcrum   13 Jul 15 07:59 
-rw-r--r-- 1 fulcrum fulcrum  807 Jul 15 07:56 .profile

• Change to the fulcrum data folder

cd /data/fulcrum

• Generate cert and key files for SSL

openssl req -newkey rsa:2048 -sha256 -nodes -x509 -days 3650 -subj "/O=Fulcrum" -keyout key.pem -out cert.pem

Expected output:

> Generating a RSA private key
> ....................+++++
> ..................................+++++
> writing new private key to 'key.pem'
> -----

• Download the custom Fulcrum banner based on MiniBolt. Create your own if you want here

wget https://raw.githubusercontent.com/minibolt-guide/minibolt/main/resources/fulcrum-banner.txt

Configuration

MiniBolt uses SSL as default for Fulcrum, but some wallets like BlueWallet do not support SSL over Tor. That's why we use TCP in configurations as well to let the user choose what he needs. You may as well need to use TCP for other reasons.

• Create a Fulcrum configuration file

nano /data/fulcrum/fulcrum.conf

• Enter the following content. Save and exit

Remember to accommodate the fast-sync parameter depending on your hardware

# MiniBolt: fulcrum configuration
# /data/fulcrum/fulcrum.conf

## Bitcoin Core settings
bitcoind = 127.0.0.1:8332
rpccookie = /data/bitcoin/.cookie

## Admin Script settings
admin = 8000

## Fulcrum server general settings
datadir = /data/fulcrum/fulcrum_db
cert = /data/fulcrum/cert.pem
key = /data/fulcrum/key.pem
ssl = 0.0.0.0:50002
tcp = 0.0.0.0:50001
peering = false

# Set utxo-cache according to your device performance,
# recommended: utxo-cache=1/2 x RAM available e.g: 4GB RAM -> utxo-cache=2000
utxo-cache = 

# Banner
banner = /data/fulcrum/fulcrum-banner.txt

Remember, if you have a slow-performance device, follow the slow device section to improve the experience of the first indexation

• Exit the fulcrum user session to return to the "admin" user session

exit

Create systemd service

Fulcrum needs to start automatically on system boot.

• As user admin, create the Fulcrum systemd unit

sudo nano /etc/systemd/system/fulcrum.service

• Enter the complete following configuration. Save and exit

# MiniBolt: systemd unit for Fulcrum
# /etc/systemd/system/fulcrum.service

[Unit]
Description=Fulcrum
Requires=bitcoind.service
After=bitcoind.service

StartLimitBurst=2
StartLimitIntervalSec=20

[Service]
ExecStart=/usr/local/bin/Fulcrum /data/fulcrum/fulcrum.conf
ExecStop=/usr/local/bin/FulcrumAdmin -p 8000 stop

User=fulcrum
Group=fulcrum

# Process management
####################
Type=exec
KillSignal=SIGINT
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

• Enable autoboot (optional)

sudo systemctl enable fulcrum

• Prepare "fulcrum" monitoring by the systemd journal and check log logging output. You can exit monitoring at any time with Ctrl-C

journalctl -fu fulcrum

Run

To keep an eye on the software movements, start your SSH program (eg. PuTTY) a second time, connect to the MiniBolt node, and log in as "admin"

• Start the service

sudo systemctl start fulcrum

Example of expected output on the first terminal with journalctl -fu fulcrum ⬇️

Fulcrum must first fully index the blockchain and compact its database before you can connect to it with your wallets. This can take up to ~1.5 - 4 days or more, depending on the hardware. Only proceed with the Blockchain explorer: BTC RPC Explorer and Desktop Wallet Section once Fulcrum is ready.

Fulcrum will now index the whole Bitcoin blockchain so that it can provide all necessary information to wallets. With this, the wallets you use no longer need to connect to any third-party server to communicate with the Bitcoin peer-to-peer network.

DO NOT REBOOT OR STOP THE SERVICE DURING THE DB CREATION PROCESS. YOU MAY CORRUPT THE FILES - in case that happens, start the sync from scratch by deleting the content of fulcrum_db folder, follow the next steps:

• With user admin, stop fulcrum

sudo systemctl stop fulcrum

• Delete the fulcrum_db folder content

sudo rm -r /data/fulcrum/fulcrum_db/*

• Start fulcrum again

sudo systemctl start fulcrum

-> You should see the logs of the Run process before again

-> The troubleshooting note could be helpful after experiencing data corruption due to a power outage during normal operation

• When you see logs like this SrvMgr: starting 3 services ..., which means that Fulcrum is fully indexed

Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.705] SrvMgr: starting 3 services ...
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.706] Starting listener service for TcpSrv 0.0.0.0:50001 ...
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.706] Service started, listening for connections on 0.0.0.0:50001
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.706] Starting listener service for SslSrv 0.0.0.0:50002 ...
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.706] Service started, listening for connections on 0.0.0.0:50002
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.707] Starting listener service for AdminSrv 127.0.0.1:8000 ...
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.707] Service started, listening for connections on 127.0.0.1:8000
Jun 09 10:28:56 minibolt Fulcrum[3345722]: [2024-06-09 10:28:56.707] <Controller> Starting ZMQ Notifier (hashblock) ...

• Ensure the service is working and listening at the default 50002 & 50001 ports and the 8000 admin port

sudo ss -tulpn | grep LISTEN | grep Fulcrum

Expected output:

tcp   LISTEN 0      50        0.0.0.0:50001      0.0.0.0:*    users:(("Fulcrum",pid=1821,fd=185))
tcp   LISTEN 0      50        0.0.0.0:50002      0.0.0.0:*    users:(("Fulcrum",pid=1821,fd=204))
tcp   LISTEN 0      50      127.0.0.1:8000       0.0.0.0:*    users:(("Fulcrum",pid=1821,fd=206))

Extras (optional)

Remote access over Tor

• Ensure that you are logged in with the user admin and edit the torrc file

sudo nano /etc/tor/torrc

• Add the following lines in the "location hidden services" section, below ## This section is just for location-hidden services ## in the torrc file. Save and exit

# Hidden Service Fulcrum TCP & SSL
HiddenServiceDir /var/lib/tor/hidden_service_fulcrum_tcp_ssl/
HiddenServiceVersion 3
HiddenServicePoWDefensesEnabled 1
HiddenServicePort 50001 127.0.0.1:50001
HiddenServicePort 50002 127.0.0.1:50002

• Reload the Tor configuration to apply changes

sudo systemctl reload tor

• Get your Onion address

sudo cat /var/lib/tor/hidden_service_fulcrum_tcp_ssl/hostname

Example of expected output:

> abcdefg..............xyz.onion

• You should now be able to connect to your Fulcrum server remotely via Tor using your hostname and port 50001 (TCP) or 50002 (SSL)

Admin Script: FulcrumAdmin

Fulcrum comes with an admin script. The admin service is used for sending special control commands to the server, such as stopping the server. You may send commands to Fulcrum using this script.

• Type the next command to see a list of possible subcommands that you can send to Fulcrum

FulcrumAdmin -h

Expected output:

usage: FulcrumAdmin [-h] -p port [-j] [-H [host]]
                  {addpeer,ban,banpeer,bitcoind_throttle...
[...]

• Type the next command to get complete server information

FulcrumAdmin -p 8000 getinfo

Get more information about this command in the official documentation section

Slow devices mode

Fulcrum configuration

• As the admin user, edit the existing fulcrum.conf file

 sudo nano /data/fulcrum/fulcrum.conf

• Uncomment the db_max_open_files parameter choosing the appropriate one for 4 GB or 8 GB of RAM depending on your hardware

## Slow device first-time start optimizations
bitcoind_timeout = 600
bitcoind_clients = 1
worker_threads = 1
db_mem = 1024.0

# 4GB RAM
#db_max_open_files = 200
# 8GB RAM
#db_max_open_files = 400

Install zram-swap

zram-swap is a compressed swap in memory and on disk and is necessary for the proper functioning of Fulcrum during the sync process using compressed swap in memory (increase performance when memory usage is high)

• With user admin, access to the "admin" home folder

cd /home/admin/

• Clone the repository of GitHub and go to the zram-swap folder

git clone https://github.com/foundObjects/zram-swap.git && cd zram-swap

• Install it

sudo ./install.sh

• Add kernel parameters to make better use of zram

sudo nano /etc/sysctl.conf

• Add next lines at the end of the file. Save and exit

vm.vfs_cache_pressure=500
vm.swappiness=100
vm.dirty_background_ratio=1
vm.dirty_ratio=50

• Then apply the changes with

sudo sysctl --system

• Restart the service

sudo systemctl restart zram-swap

• Make sure zram was correctly installed and prioritized (+ Priority than swap)

sudo cat /proc/swaps

Example of expected output:

Filename               Type            Size            Used            Priority
/swap.img              file            4194300         0               -2
/dev/zram0             partition       10055452        368896          15

Upgrade

Follow the complete Download and set up Fulcrum section replacing the environment variable "VERSION=x.xx" value for the latest if it has not been already changed in this guide.

• Restart the service to apply the changes

sudo systemctl restart fulcrum

• Check logs and pay attention to the next log if that attends to the new version installed

journalctl -fu fulcrum

Example of expected output:

Jul 28 12:20:13 minibolt Fulcrum[181811]: [2022-07-28 12:20:13.064] Fulcrum 1.9.1 (Release a5a53cf) - Wed Dec 21, 2022 15:35:25.963 UTC - starting up ...
[...]

Uninstall

Uninstall service

• Ensure you are logged in with the user admin, stop fulcrum

sudo systemctl stop fulcrum

• Disable autoboot (if enabled)

sudo systemctl disable fulcrum

• Delete the service

sudo rm /etc/systemd/system/fulcrum.service

Delete user & group

• Delete the fulcrum user. Don't worry about userdel: fulcrum mail spool (/var/mail/fulcrum) not found output, the uninstall has been successful

sudo userdel -rf fulcrum

Delete data directory

• Delete fulcrum directory

sudo rm -rf /data/fulcrum/

Uninstall binaries

• Delete the binaries installed

sudo rm /usr/local/bin/Fulcrum && sudo rm /usr/local/bin/FulcrumAdmin

Uninstall Tor hidden service

• Ensure that you are logged in with the user admin and delete or comment the following lines in the "location hidden services" section, below "## This section is just for location-hidden services ##" in the torrc file. Save and exit

sudo nano /etc/tor/torrc

# Hidden Service Fulcrum TCP & SSL
#HiddenServiceDir /var/lib/tor/hidden_service_fulcrum_ssl/
#HiddenServiceVersion 3
#HiddenServicePoWDefensesEnabled 1
#HiddenServicePort 50001 127.0.0.1:50001
#HiddenServicePort 50002 127.0.0.1:50002

• Reload the torrc config

sudo systemctl reload tor

Uninstall FW configuration

• Ensure you are logged in with the user admin, display the UFW firewall rules, and note the numbers of the rules for Fulcrum (e.g., X and Y below)

sudo ufw status numbered

Expected output:

> [Y] 50001       ALLOW IN    Anywhere          # allow Fulcrum TCP from anywhere
> [X] 50002       ALLOW IN    Anywhere          # allow Fulcrum SSL from anywhere

• Delete the rule with the correct number and confirm with "yes"

sudo ufw delete X

Uninstall the Zram

• Ensure you are logged in with the user admin, navigate to the zram-swap folder, and uninstall

cd /home/admin/zram-swap

sudo ./install.sh --uninstall

sudo rm /etc/default/zram-swap

sudo rm -rf /home/admin/zram-swap

• Make sure that the change was done

sudo cat /proc/swaps

Expected output:

Filename            Type                Size           Used    Priority
/var/swap           file               102396            0        -2

Port reference

Port	Protocol	Use
50001	TCP	Default port
50002	TCP (SSL)	Encrypted port
8000	TCP	Admin port