Skip to main content

Running a fullnode/vfn/validator node with docker

WIP

This page is a work in progress. Tread carefully.

The canonical way of running the standard 0L setup, is to have two machines, one for the validator node, and one for the fullnode that acts as the gateway of the validator to the world (see VFN here).

This page describes how to setup a node with docker, which provides a convenient way to hit the ground running without managing packages and installation on the host machine.

In addition, a validator can take an advantage of this configuration and run both the validator node and the VFN on the same machine. This would require a second network interface with a new IPv4 address. See further details below.

Prerequisites

This guide would not cover those

  • Ubuntu 22.04
  • Docker
  • docker compose
  • sudo access

Build the libra node docker image (both for single or double IPv4)

Place this Dockerfile somewhere in your system (~/workspace/SOME_NAME/Dockerfile):

# Dockerfile

# Use Debian 11 image as the base for the build stage
FROM debian:11 as build

# Set the PATH environment variable to include the Rust Cargo bin directory
ENV PATH="/root/.cargo/bin:${PATH}"

# Install system dependencies required for building the project
# These include compilers, development tools, libraries, and other utilities
RUN apt-get update -y -q && apt-get install -y -q \
build-essential \
curl \
cmake \
clang \
git \
libgmp3-dev \
libssl-dev \
llvm \
lld \
pkg-config \
ca-certificates \
update-ca-certificates \
&& rm -rf /var/lib/apt/lists/*


# Install Rust using rustup and set the default toolchain to stable
RUN curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain stable -y

# Install Rust packages with Cargo
RUN cargo install toml-cli # Command-line tool to manipulate TOML files
RUN cargo install sccache # Shared Compilation Cache to speed up recompiles

# Arguments that can be overridden at build time
ARG REPO=https://github.com/0LNetworkCommunity/libra-framework.git
ARG BRANCH=main

# Set the working directory to /root
WORKDIR /root

# Clone the specified branch or release tag from the given repository
RUN echo "Checking out '${BRANCH}' from '${REPO}' ..." \
&& git clone ${REPO} \
&& cd libra-framework \
&& git fetch --all && git checkout ${BRANCH} \
&& echo "Commit hash: $(git rev-parse HEAD)"

# Set the working directory to the cloned repository
WORKDIR /root/libra-framework

# Build the specified Rust packages as release binaries
RUN cargo build --release \
-p libra \
-p libra-genesis-tools \
-p libra-txs \
-p diem-db-tool

# Start a new, final image to reduce size using Debian 11 slim variant
FROM debian:11-slim as production

# Copy the built binaries from the 'build' stage to the 'production' image
COPY --from=build [ \
"/root/libra-framework/target/release/libra", \
"/root/libra-framework/target/release/libra-genesis-tools", \
"/root/libra-framework/target/release/libra-txs", \
"/root/libra-framework/target/release/diem-db-tool", \
"/usr/local/bin/"]

In the same folder you placed the Dockerfile, run the following commands to build the image

docker build -t openlibra:main .

This will take several minutes, depends on your machine. A community image might be available in the future from docker hub. There is a value also in building your own image and not relying on a centralized image.

Ensure image was built successfully

docker images | grep openlibra

openlibra main b64dbca39f51 9 minutes ago 153MB

Single IP setup

Create the docker-compose file

Place the following docker-compose.yaml file in the same folder as the Dockerfile:

# docker-compose.yaml

########## Defaults #############
x-defaults: &defaults
image: "openlibra:main"
restart: "on-failure"
pid: host
ulimits:
nproc: 500000
nofile: 500000
volumes:
- "node_data:/root/.libra"

x-util-defaults: &util-defaults
<<: *defaults
restart: "no"
command: [ "tail", "-f", "/dev/null" ]

version: "3.8"
services:
########## Main services #############

fullnode:
<<: *defaults
container_name: "0l-fullnode"
cpuset: "0"
command:
[
"libra",
"node",
"--config-path",
"/root/.libra/fullnode.yaml"
]
ports:
- "6180:6180"
- "6181:6181"
- "6182:6182"
- "8080:8080"
- "9101:9101"

validator:
<<: *defaults
container_name: "0l-validator"
cpuset: "0"
command:
[
"libra",
"node",
"--config-path",
"/root/.libra/validator.yaml"
]
ports:
- "6180:6180"
- "6181:6181"
- "6182:6182"
- "8080:8080"
- "9101:9101"

vfn:
<<: *defaults
container_name: "0l-vfn"
cpuset: "0"
command:
[
"libra",
"node",
"--config-path",
"/root/.libra/vfn.yaml"
]
ports:
- "6180:6180"
- "6181:6181"
- "6182:6182"
- "8080:8080"
- "9101:9101"

tower: # needs mnemonic injection
<<: *defaults
container_name: "0l-tower"
cpuset: "1"
stdin_open: true
tty: true
command:
[
"libra",
"tower",
"start",
]

shell:
<<: *util-defaults
container_name: 0l-shell

volumes:
node_data:
driver: local
driver_opts:
type: none
o: bind
device: "~/.libra"

Ensure your configuration files are in place.

Depends on which service you wish to run (fullnode/validator/vfn), you'd need to place the relevant yaml files in the ~/.libra folder.

Run the following commands based on your needs:

libra config fullnode-init

# or

libra config validator-init

# or

libra config validator-init --vfn

Note that you'd need to replace your user home path in the template default data_dir, and the genesis_file_location to point to /root/.libra/...rest_of_path_unchanged

Here is an example of modified fullnode.yaml relevant entries:

...

base:
data_dir: '/root/.libra/data'

execution:
genesis_file_location: '/root/.libra/genesis/genesis.blob'
...

Start the desired service with docker compose

docker compose up -d fullnode

# or

docker compose up -d validator

# or vfn or tower, etc.

Ensure the node is running by examining the updated ledger_version over few seconds

watch 'curl localhost:8080/v1/ | jq'

Or check the container logs

docker compose logs -f --tail 50 fullnode

# or replace fullnode with validator or vfn
If the version is 0 or you're having connectvitiy issues

Do the manual DB restore as described in the Restore section (TODO: link) and then clean and sync (TODO: link)

Run tower

docker compose run tower

Create the docker-compose file (double IP)

WIP Beyond this line

Dual NIC setup

Part 1: Configure second IP

Using Netplan (default ubuntu network manager), edit your /etc/netplan/01-netcfg.yaml

Identify your ethernet id (enp5s0 in this example), add the second IP to its addresses list

network:
version: 2
renderer: networkd
ethernets:
enp5s0:
addresses:
- 188.primary.ip.here/32
- 188.secondary.ip.here/32 # <--- add this line
- primary:ip:v:6/64
routes:
- on-link: true
to: 0.0.0.0/0
via: your.ip.subnet.mask
- to: default
via: fe80::1
nameservers:
addresses:
- some.ip.v.4
- some.ip.v.6

Apply changes sudo netplan apply

Confirm changes ip addr

Under the modified interface, you should see the second IP address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether c8:7f:tag:tag:tag brd ff:ff:ff:ff:ff:ff
############
inet your.primary.ip.here/32 scope global enp5s0
valid_lft forever preferred_lft forever
inet your.secondary.ip.here/32 scope global enp5s0 # <--- make sure second IP appears here
valid_lft forever preferred_lft forever
############
inet6 primary:ip:v:6/64 scope global
valid_lft forever preferred_lft forever