Hub to Site
Crux VPN enables flexible network topologies, allowing you to securely forward traffic from one site to another through various intermediate servers in a private Crux network.
Consider the following example, where we've set up a Crux VPN hub on a server that's publicly accessible over the Internet at 198.51.100.51; and allows encrypted connections through its private Crux network from remote endpoints like a mobile phone on a public cellular network, or to other remote sites or individual cloud servers across the Internet; plus allows encrypted connections from other private networks deep within its own site:
Crux Hub
The Crux Hub in this scenario would listen on public UDP port 51820 for encrypted WireGuard traffic and TCP port 8887 for Arqit SKA-Platform™ (SKA-P) peering, to which the other nodes in the Crux network would connect in order to set up their encrypted tunnels. The Crux Hub would route traffic on its 10.10.10.0/24 private Crux network to other Crux nodes; as well as 192.168.200.0/24 to and from the Site Gateway, and 192.168.100.0/31 to and from the Single Device Gateway.
Traffic from the Inbound NAT LAN (192.168.0.0/24) would be forwarded by the Inbound NAT Server to the Crux Hub using its own Crux address of 10.10.10.30; and traffic to the Outbound NAT LAN (192.168.1.0/24) would be forwarded using the Outbound NAT Server's Crux address of 10.10.10.40, and translated by the Outbound NAT Server to the appropriate destination on its LAN.
Firewall on the Crux Hub
The Crux Hub's own host-based firewall, and any firewalls between it and other nodes that connect directly to the Crux Hub, need to allow inbound access in the Crux Hub's direction to UDP port 51820 and TCP port 8887.
The Crux Hub's firewall can be used for fine-grained access control within the Crux network (for example, to prevent any hosts except the Site Webserver from accessing the Backend Server) -- in which case we'd need to configure the forwarding rules for the firewall manually. Additionally, we'd need to enable the net.ipv4.conf.all.forwarding kernel parameter on the Crux Hub (like by running the sysctl -w net.ipv4.conf.all.forwarding=1 command on it). See the Firewall Rules page for more details.
If fine-grained access control on the hub is not needed, the Crux VPN web UI can be used to set up firewall rules that allow unrestricted forwarding between nodes in the Crux network. We can do this on the Hosts > Crux Hub > Interfaces > Edit page, by opening the Advanced Properties section on this page, and selecting the Internal within WireGuard network option from Forwarding dropdown. (This is the default setting if we create the Crux Hub as a simple server in the web UI.)
Site Gateway
The Site Gateway in this example is a simple server with a LAN IP address of 192.168.200.234 on the 192.168.200.0/24 subnet at a remote site. We want it to forward traffic both ways between its LAN (192.168.200.0/24) and the Crux network (10.10.10.0/24), as well as to the Single Device network (192.168.100.100/31). For example, we want to allow the Mobile Phone at 10.10.10.10 to access the Site Webserver at 192.168.200.22; and allow the Site Webserver at 192.168.200.22 to access the Backend Server at 10.10.10.20.
We can start setting this up in the Crux VPN web UI by creating a host record for the Site Gateway, adding it as a simple client to the Crux Hub host. When we create it, we assign it a WireGuard address of 10.10.10.200, which will be its address on the private Crux network. Once we install the Crux VPN agent on the Site Gateway host, the Site Gateway server itself will be able to access any of the other nodes connected directly to the private Crux network via the Crux Hub.
However, by default, the other hosts on the Site Gateway's LAN won't be able to access the Crux network, and the other nodes in the Crux network won't be able to access the Site Gateway's LAN (and also the Single Device network won't be accessible to the Site Gateway or its LAN). To change that, we need to do four things:
- Add routes for the LAN network from the hub to the gateway
- Add routes for the other networks from the gateway to the hub
- Configure the gateway firewall to allow forwarding
- Add routes for the Crux network (and other networks) to the LAN router (or to individual hosts on the LAN)
Routes from Crux Hub to Site Gateway
To allow the Crux Hub to send traffic for the Site Gateway's LAN (192.168.200.0/24) via the Site Gateway, we need to configure the endpoint from the Crux Hub to the Site Gateway in the Crux VPN web UI.
This can be accomplished in the web UI by navigating to the Hosts > Crux Hub > Interfaces > crux0 > Endpoints > to Site Gateway > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.200/32 to 10.10.10.200/32, 192.168.200.0/24. Once this change is applied, the Crux Hub will route traffic sent through the Crux network for the Site Gateway's LAN via the Site Gateway.
Routes from Site Gateway to Crux Hub
To allow the Site Gateway to send traffic for the Single Device's subnet (192.168.100.100/31) via the Single Device Gateway, we need to configure the endpoint from the Site Gateway to the Crux Hub in the Crux VPN web UI (the reverse direction of the first step).
This can be accomplished in the web UI by navigating to the Hosts > Site Gateway > Interfaces > crux0 > Endpoints > to Crux Hub > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.0/24 to 10.10.10.0/24, 192.168.100.100/31. Once this change is applied, the Site Gateway will route traffic sent from the Site Gateway's LAN to the Single Device via the Crux Hub.
Firewall on Site Gateway
To allow forwarding of traffic between the Crux network and the Site Gateway's LAN, we need add some firewall rules to the Site Gateway.
If the Site Gateway's firewall doesn't have and doesn't need any custom firewall rules to restrict access between the Crux network and the Site Gateway's LAN, then we can use the web UI to set up these firewall rules. We can do this on the Hosts > Site Gateway > Interfaces > Edit page, by opening the Advanced Properties section on this page, and selecting the All to and from WireGuard network option from Forwarding dropdown.
If the Site Gateway has or needs custom firewall rules, then we would need to customize the firewall manually. If the Site Gateway was a Linux server using iptables, we might add the following two rules to its FORWARD chain for this:
iptables -I FORWARD -i crux0 -j ACCEPT
iptables -I FORWARD -o crux0 -j ACCEPT
Additionally, we'd need to make sure that we had enabled the net.ipv4.conf.all.forwarding kernel parameter on the Site Gateway (like by running the sysctl -w net.ipv4.conf.all.forwarding=1 command on it). See the Firewall Rules page for more details.
Routes on Site Gateway LAN
To allow the hosts on the Site Gateway's LAN to send traffic to the Crux network and the Single Device through the Site Gateway, we need to configure the LAN router at the site with the appropriate routes. Alternatively, we could configure each individual host on the Site Gateway's LAN that needed to send traffic through the Crux network with the appropriate routes.
On Site Gateway hosts that were running Linux, we could add the necessary routes to them by running the following commands (where 192.168.200.234 is the LAN address for the Site Gateway, and eth1 is the name of LAN network interface on the particular Linux box on which the command is run):
ip route add 10.10.10.0/24 via 192.168.200.234 dev eth1
ip route add 192.168.100.100/31 via 192.168.200.234 dev eth1
Single Device Gateway
The Single Device Gateway in this example is a simple server (which could be as simple as a Raspberry Pi or other single-board computer) to which the Single Device is connected via Ethernet. The Single Device Gateway allows us to plug a proprietary device into the Crux network that cannot run the Crux VPN agent itself (either for administrative, legal, or technical reasons).
We'd set up the Ethernet connection between the Single Device Gateway and the Single Device with a simple point-to-point subnet of 192.168.100.100/31, assigning 192.168.100.100 to the Single Device Gateway and 192.168.100.101 to the Single Device. We want the Single Device Gateway to forward traffic both ways between this point-to-point subnet (192.168.100.100/31) and the Crux network (10.10.10.0/24), as well as to the Site Gateway LAN (192.168.200.0/24). For example, we want to allow the Mobile Phone at 10.10.10.10 to access the Single Device at 192.168.100.101; and allow the Single Device at 192.168.100.101 to access the Site Webserver at 192.168.200.22.
We can start setting this up in the Crux VPN web UI by creating a host record for the Single Device Gateway, adding it as a simple client to the Crux Hub host. When we create it, we assign it a WireGuard address of 10.10.10.100, which will be its address on the private Crux network. Once we install the Crux VPN agent on the Single Device Gateway host, the Single Device Gateway itself will be able to access any of the other nodes connected to this private Crux network via the Crux Hub.
However, by default, the Single Device connected to the Single Device Gateway won't be able to access the Crux network; and the other nodes in the Crux network won't be able to access the Single Device (and also the Site Gateway's LAN won't be accessible to the Single Device). To change that, we need to do four things:
- Add routes for the single device from the hub to the gateway
- Add routes for the other networks from the gateway to the hub
- Configure the gateway firewall to allow forwarding
- Add routes for the Crux network (and other networks) to the single device
Routes from Crux Hub to Single Device Gateway
To allow the Crux Hub to send traffic for the Single Device (192.168.100.101) via the Single Device Gateway, we need to configure the endpoint from the Crux Hub to the Single Device Gateway in the Crux VPN web UI.
This can be accomplished in the web UI by navigating to the Hosts > Crux Hub > Interfaces > crux0 > Endpoints > to Single Device Gateway > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.100/32 to 10.10.10.100/32, 192.168.100.100/31. Once this change is applied, the Crux Hub will route traffic sent through the Crux network for the Single Device via the Single Device Gateway.
Routes from Single Device Gateway to Crux Hub
To allow the Single Device Gateway to send traffic for the Site Gateway's LAN (192.168.200.0/24) via the Site Gateway, we need to configure the endpoint from the Single Device Gateway to the Crux Hub in the Crux VPN web UI (the reverse direction of the first step).
This can be accomplished in the web UI by navigating to the Hosts > Single Device Gateway > Interfaces > crux0 > Endpoints > to Crux Hub > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.0/24 to 10.10.10.0/24, 192.168.200.0/24. Once this change is applied, the Single Device Gateway will route traffic sent from the Single Device to the Site Gateway's LAN via the Crux Hub.
Firewall on Single Device Gateway
To allow forwarding of traffic between the Crux network and the Single Device, we need add some firewall rules to the Single Device Gateway.
If the Single Device Gateway's firewall doesn't have and doesn't need any custom firewall rules to restrict access between the Crux network and any of its other networks, then we can use the web UI to set up these firewall rules. We can do this on the Hosts > Single Device Gateway > Interfaces > Edit page, by opening the Advanced Properties section on this page, and selecting the All to and from WireGuard network option from Forwarding dropdown.
If the Single Device Gateway has or needs custom firewall rules, then we would need to customize the firewall manually. If the Single Device Gateway was a Linux server using iptables, we might add the following two rules to its FORWARD chain for this:
iptables -I FORWARD -i crux0 -j ACCEPT
iptables -I FORWARD -o crux0 -j ACCEPT
Additionally, we'd need to make sure that we had enabled the net.ipv4.conf.all.forwarding kernel parameter on the Single Device Gateway (like by running the sysctl -w net.ipv4.conf.all.forwarding=1 command on it). See the Firewall Rules page for more details.
Routes on Single Device
To allow the Single Device to send traffic to the Crux network and the Site Gateway's LAN through the Single Device Gateway, we need to configure the Single Device with the appropriate routes.
If the Single Device is running Linux, we could add the necessary routes to it by running the following commands (where 192.168.100.100 is the address of the Single Device Gateway on its point-to-point Ethernet link, and eth1 is the name of the Ethernet interface to that link on the Single Device):
ip route add 10.10.10.0/24 via 192.168.100.100 dev eth1
ip route add 192.168.200.0/24 via 192.168.100.100 dev eth1
Outbound NAT Server
The Outbound NAT Server in this example is a simple server with a LAN IP address of 192.168.1.23 on the 192.168.1.0/24 subnet at a remote site. We want it to forward traffic from the Crux network (10.10.10.0/24), plus the Site Gateway's LAN and (192.168.200.0/24) and Single Device network (192.168.100.100/31), to the Outbound NAT Target at TCP port 80 of 192.168.1.42, using the Outbound NAT Server's own Crux address of 10.10.10.40 (and a different TCP port of 4000). This will allow devices connected to the Crux network, such as the Mobile Phone (10.10.10.10) or Single Device (192.168.100.101) to access the Outbound NAT Target using a URL of http://10.10.10.40:4000.
This technique allows outbound connections from the Crux network to be forwarded on to one or more other network services that the Outbound NAT Server can access -- and that other nodes in the Crux network would otherwise not be able to access. The advantage of using a NAT server instead of a full site or device gateway is that it requires no routing changes for its own LAN.
The disadvantage is that it allows only one-way access to a pre-specified set of services: the Outbound NAT Target cannot initiate connections to nodes on the Crux network -- it can only respond back to connections initiated through the Outbound NAT Server. Also, the Outbound NAT Target is not able to check or log the original source of its connections -- from its perspective, all the traffic from the Crux network appears to being coming directly from the Outbound NAT Server, rather than individual hosts in the Crux network.
We can start setting this up in the Crux VPN web UI by creating a host record for the Outbound NAT Server, adding it as a simple client to the Crux Hub host. When we create it, we assign it a WireGuard address of 10.10.10.40, which will be its address on the private Crux network. Once we install the Crux VPN agent on the Outbound NAT Server host, the Outbound NAT server itself will be able to access any of the other nodes connected to the private Crux network via the Crux Hub.
Routes from Outbound NAT Server to Crux Hub
To allow the Outbound NAT Server to send traffic for the Site Gateway's LAN (192.168.200.0/24) via the Site Gateway, and send traffic for the Single Device subnet (192.168.100.100/31) via the Single Device, we need to configure the endpoint from the Outbound NAT Server to the Crux Hub in the Crux VPN web UI.
This can be accomplished in the web UI by navigating to the Hosts > Outbound NAT Server > Interfaces > crux0 > Endpoints > to Crux Hub > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.0/24 to 10.10.10.0/24, 192.168.100.100/31, 192.168.200.0/24. Once this change is applied, the Outbound NAT Server will route return traffic sent from the Outbound NAT Target to the Site Gateway's LAN or the Single Device via the Crux Hub.
Firewall on Outbound NAT Server
To enable the Outbound NAT Server to forward connections to the Outbound NAT Target, we need to configure its firewall both to allow the forwarded connections, and to translate the addresses of those connections.
If the Outbound NAT Server was a Linux server using iptables, we would set up a ruleset on the server like this:
iptables-restore <<'EOF'
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
# allow SSH from anywhere to self
-A INPUT -p tcp --dport 22 -j ACCEPT
# forward return traffic of already-established connections
-A FORWARD -m state --state INVALID -j DROP
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# forward connections initiated from Crux network to Outbound NAT Target
-A FORWARD -i crux0 -d 192.168.1.42 -p tcp --dport 80 -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# translate destination address of connections originally to own TCP port 4000
# instead to Outbound NAT Target TCP port 80
-A PREROUTING -p tcp --dport 4000 -j DNAT --to-destination 192.168.1.42:80
# translate source address of connections forwarded out LAN interface
# instead to use own LAN address
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
EOF
The key elements of this ruleset are the rule in its filter table that allows connections to be forwarded to the Outbound NAT Target:
-A FORWARD -i crux0 -d 192.168.1.42 -p tcp --dport 80 -j ACCEPT
And the rule in its nat table that translates the destination address and port of connections originally to 10.10.10.40:4000 instead to 192.168.1.42:80:
-A PREROUTING -p tcp --dport 4000 -j DNAT --to-destination 192.168.1.42:80
And the rule in its nat table that translates the source address of connections forwarded out its LAN interface (which originally might be from 10.10.10.10 or 192.168.100.101) instead to use its own LAN address of 192.168.1.23:
-A POSTROUTING -o eth0 -j MASQUERADE
In addition to these firewall rules, we'd need to make sure that we had enabled the net.ipv4.conf.all.forwarding kernel parameter on the Outbound NAT Server (like by running the sysctl -w net.ipv4.conf.all.forwarding=1 command on it). See the Firewall Rules page for more details.
Inbound NAT Server
The Inbound NAT Server in this example is a simple server with a LAN IP address of 192.168.0.123 on the 192.168.0.0/24 subnet in the midst of a larger site. We want it to forward traffic from computers on its LAN subnet (192.168.0.0/24) to the Site Webserver (192.168.200.22) at TCP port 80, using the Inbound NAT Server's own LAN address of 192.168.0.123 (and a different TCP port of 3000). This will allow devices on the LAN but not on the Crux network, such as the Inbound NAT Client (192.168.0.31), to access the remote Site Webserver through the Crux network using a URL of http://192.168.0.123:3000.
This technique allows inbound connections from clients not on the Crux network to be forwarded on to one or more other network services reachable only through the Crux network. The advantage of using a NAT server instead of a full site or device gateway is that it requires no routing changes for its own LAN.
The disadvantage is that it allows only one-way access to a pre-specified set of services: other nodes in the Crux network cannot initiate access to any hosts on the Inbound NAT Server LAN -- they can only respond back to connections initiated through the Inbound NAT Server. Also, the Crux network loses visibility to the original source of the connections forwarded through the Inbound NAT Server -- from the perspective of hosts in the Crux network, all the traffic from the Inbound NAT Server LAN appears to being coming directly from the Inbound NAT Server, rather than the individual hosts on its LAN.
We can start setting this up in the Crux VPN web UI by creating a host record for the Inbound NAT Server, adding it as a simple client to the Crux Hub host. When we create it, we assign it a WireGuard address of 10.10.10.30, which will be its address on the private Crux network. Once we install the Crux VPN agent on the Inbound NAT Server host, the Inbound NAT server itself will be able to access any of the other nodes connected to this private Crux network via the Crux Hub.
Routes from Inbound NAT Server to Crux Hub
To allow the Inbound NAT Server to send traffic for the Site Gateway's LAN (192.168.200.0/24) via the Site Gateway (and send traffic for the Single Device subnet, 192.168.100.100/31, via the Single Device if we wanted to enable that in the future) we need to configure the endpoint from the Inbound NAT Server to the Crux Hub in the Crux VPN web UI.
This can be accomplished in the web UI by navigating to the Hosts > Inbound NAT Server > Interfaces > crux0 > Endpoints > to Crux Hub > Edit page. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.0/24 to 10.10.10.0/24, 192.168.100.100/31, 192.168.200.0/24. Once this change is applied, the Inbound NAT Server will route traffic sent from the Inbound NAT Client to the Site Gateway's LAN (or the Single Device) via the Crux Hub.
Firewall on Inbound NAT Server
To enable the Inbound NAT Server to forward connections from the Inbound NAT Client, we need to configure its firewall both to allow the forwarded connections, and to translate the addresses of those connections.
If the Inbound NAT Server was a Linux server using iptables, we would set up a ruleset on the server like this:
iptables-restore <<'EOF'
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
# allow SSH from anywhere to self
-A INPUT -p tcp --dport 22 -j ACCEPT
# forward return traffic of already-established connections
-A FORWARD -m state --state INVALID -j DROP
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# forward connections initiated from LAN to Site Webserver
-A FORWARD -i eth0 -d 192.168.200.22 -p tcp --dport 80 -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# translate destination address of connections originally to own TCP port 3000
# instead to Site Webserver TCP port 80
-A PREROUTING -p tcp --dport 3000 -j DNAT --to-destination 192.168.200.22:80
# translate source address of connections forwarded out Crux interface
# instead to use own Crux address
-A POSTROUTING -o crux0 -j MASQUERADE
COMMIT
EOF
The key elements of this ruleset are the rule in its filter table that allows connections to be forwarded from its LAN to the Site Webserver:
-A FORWARD -i eth0 -d 192.168.200.22 -p tcp --dport 80 -j ACCEPT
And the rule in its nat table that translates the destination address and port of connections originally to 192.168.0.123:3000 instead to 192.168.200.22:80:
-A PREROUTING -p tcp --dport 3000 -j DNAT --to-destination 192.168.200.22:80
And the rule in its nat table that translates the source address of connections forwarded out its Crux interface (which originally might be from 192.168.0.31) instead to use the Inbound NAT Server's own Crux address of 10.10.10.30:
-A POSTROUTING -o crux0 -j MASQUERADE
In addition to these firewall rules, we'd need to make sure that we had enabled the net.ipv4.conf.all.forwarding kernel parameter on the Inbound NAT Server (like by running the sysctl -w net.ipv4.conf.all.forwarding=1 command on it). See the Firewall Rules page for more details.
Other Nodes
All other nodes on the Crux network, such as the Mobile Phone and the Backend Server, will be able to access nodes directly on the Crux network as soon as they are sent up as simple clients.
However, to allow them to access the Site Gateway's LAN (192.168.200.0/24) through the Site Gateway, and the Single Device subnet (192.168.100.100/31) through the Single Device Gateway, we need to configure their endpoints from their own host to the Crux Hub in the Crux VPN web UI.
For example with the Backend Server, we'd navigate to the Hosts > Backend Server > Interfaces > crux0 > Endpoints > to Crux Hub > Edit page in the web UI. On that page, we'd change the Allowed IPs field from its default value of 10.10.10.0/24 to 10.10.10.0/24, 192.168.100.100/31, 192.168.200.0/24. Once this change is applied, the Backend Server will be able to route traffic intended for the Site Gateway's LAN or the Single Device via the Crux Hub.
Note that we do not have to make any changes in the reverse direction (Hosts > Crux Hub > Interfaces > crux0 > Endpoints > to Backend Server) -- the Allowed IPs field of the Crux Hub's endpoints to simple clients will default to the client's own Crux address (like 10.10.10.20/32 for the Backend Server), which is all that is needed for normal Crux nodes.
Endpoint Defaults
To avoid having to make any AllowedIPs changes at all for normal Crux nodes when we add them, we can configure the default AllowedIPs value for new endpoints to the Crux Hub by clicking on the "Defaults" icon of the "Endpoints" panel on the Crux Hub's interface page (Hosts > Crux Hub > Interfaces > crux0). This will bring up a page ("Crux Hub Defaults") where we can configure the default settings of new endpoints to the Crux Hub.
On the Crux Hub Defaults page, click the "Edit" icon of the "Defaults for Remote Endpoints" panel, and customize the Allowed IPs field as appropriate. For our example scenario, this is what we'd use for the Allowed IPs field:
10.10.10.0/24
192.168.100.100/31
192.168.200.0/24
This would ensure that any new simple clients we add to the Crux Hub will be able to route traffic to the Site Gateway LAN and Single Device networks via the Crux Hub by default, with no additional configuration necessary.