Skip to content

fix(diagram): white background and terminal server at 172.22.0.30 #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mgheorghe wants to merge 21 commits into
base: main
Choose a base branch
from
Open

fix(diagram): white background and terminal server at 172.22.0.30 #116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
0361870
chore(mgmt): sync dhcpd.conf with live config on mgmt
mgheorghe Oct 19, 2026
9441f0d
fix(mgmt): correct tor dhcp reservation
mgheorghe Oct 19, 2026
7469781
fix(mgmt): drop duplicate empty subnet declaration
mgheorghe Oct 19, 2026
5b491d5
feat(mgmt): reserve BlueField-3 DPU and BMC addresses
mgheorghe Oct 19, 2026
e8dc540
feat(mgmt): move DH9 DPU BMC to 172.22.4.9
mgheorghe Oct 19, 2026
8470a8f
feat(mgmt): move DH9 shared-NIC BMC address to 172.22.2.109
mgheorghe Oct 19, 2026
7122586
feat(mgmt): enable terminal server reservation at 172.22.0.30
mgheorghe Oct 19, 2026
b9507de
fix(mgmt): give CN106 MCU its real MAC at 172.22.4.3
mgheorghe Oct 19, 2026
360084d
fix(mgmt): update Intel IPU IMC MAC in dh4dypimc
mgheorghe Oct 19, 2026
f915095
feat(dh4): stop DHCP on second onboard port
mgheorghe Oct 19, 2026
d96c22f
fix(mgmt): correct DH4 IPU to MacKay Peak E2100, Mount Evans to DH9
mgheorghe Oct 19, 2026
d36f881
fix(tor): sync arista.config with the live running-config
mgheorghe Oct 19, 2026
78c8fca
feat(tor): put the he.net public /27 on its own VLAN 10
mgheorghe Oct 19, 2026
b6fcff6
feat(mgmt): bind isc-dhcp-server to the lab NIC only
mgheorghe Oct 19, 2026
cf445cc
feat(tor): filter internet access to the mgmt BMC on Et1
mgheorghe Oct 19, 2026
e86db25
feat(mgmt): add fail2ban sshd jail
mgheorghe Oct 19, 2026
7ee801a
docs(inventory): refresh ips.md and ansible inventory from the 2026-1…
mgheorghe Oct 19, 2026
68f7b06
chore: shorten comments and drop the pasted MAC table from dhcpd.conf
mgheorghe Oct 19, 2026
f8b5f25
docs(ips): align tables for markdownlint MD060
mgheorghe Oct 19, 2026
eb21486
docs(diagram): update rack drawing and TS console map from the 2026-1…
mgheorghe Oct 19, 2026
696f6e0
fix(diagram): white background and terminal server at 172.22.0.30
mgheorghe Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 60 additions & 44 deletions Documentation/ips.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# IP Allocations

Proposal is to use 172.22.0.0/16 network for all ip addresing in the lab, and have 1 gateway that will handle internet access and be entry point for access in the lab.
The lab uses 172.22.0.0/16; mgmt (172.22.0.1) is the NAT gateway and VPN entry point.

Last verified 2026-10-19.

## Code duplication

:exclamation: with [ansible inventory](../ansible/inventory)
:exclamation: with [ansible inventory](../ansible/inventory) and
[dhcpd.conf](../hardware/mgmt/fs/etc/dhcp/dhcpd.conf) (the `dhcpd host` column below)

## Device types ip allocation

| IR Range | Purpose |
| IP Range | Purpose |
| --------------- | --------------------------------------------------------- |
| 172.22.0.0/24 | All management devices like, gateway, kvm, ts, pdu, ... |
| 172.22.1.0/24 | All servers, dpu hosts, test, ... |
Expand All @@ -20,44 +23,57 @@ Proposal is to use 172.22.0.0/16 network for all ip addresing in the lab, and ha

## Per Device IP allocation

| IPv4 | IPv6 | DEVICE | Helpful URLs |
| ---------------------- | ---- | ----------------- | ------------------------------------------------------- |
| public ip / 172.22.0.1 | | gateway/mgmt | <http://172.22.0.1:9091/> and <http://172.22.0.1:3000/> |
| 172.22.0.5 | | ToR | |
| 172.22.0.10 | | Network PDU | |
| 172.22.0.20 | | KVM | |
| 172.22.0.30 | | TS | |
| 172.22.0.226 | | F5 VPN Floating | |
| 172.22.0.227 | | F5 BIG-IP-01 | |
| 172.22.0.228 | | F5 VPN Floating | |
| | | | |
| 172.22.1.1 | | DPU Host 1 | |
| 172.22.2.1 | | DH1 iDRAC | <https://172.22.2.1/redfish/v1/> |
| 172.22.3.1 | | AMD DSC200 | |
| | | | |
| 172.22.1.2 | | DPU Host 2 | |
| 172.22.2.2 | | DH2 iLO | <https://172.22.2.2/redfish/v1/> |
| 172.22.3.2 | | Nvidia BF2 | |
| 172.22.4.2 | | Nvidia BF2 bmc | <https://172.22.4.2/redfish/v1/> |
| | | | |
| 172.22.1.3 | | DPU Host 3 | |
| 172.22.2.3 | | DH3 iLO | <https://172.22.2.3/redfish/v1/> |
| 172.22.3.3 | | Marvell CN106 | |
| 172.22.4.3 | | Marvell CN106 bmc | <https://172.22.4.3/redfish/v1/> |
| | | | |
| 172.22.1.4 | | DPU Host 4 | |
| 172.22.2.4 | | DH4 iDRAC | <https://172.22.2.4/redfish/v1/> |
| 172.22.4.4 | | Intel DYP imc | |
| | | | |
| 172.22.1.5 | | DPU Host 5 | |
| 172.22.2.5 | | DH5 iDRAC | <https://172.22.2.5/redfish/v1/> |
| 172.22.3.5 | | Nvidia BF3 | |
| 172.22.4.5 | | Nvidia BF3 bmc | <https://172.22.4.5/redfish/v1/> |
| | | | |
| 172.22.1.9 | | DPU Host 9 | |
| 172.22.2.9 | | DH9 ??? | <https://172.22.2.9/redfish/v1/> |
| | | | |
| 172.22.1.100 | | Tgen1 | |
| 172.22.2.100 | | Tgen1 bmc | <https://172.22.2.100/redfish/v1/> |
| | | | |
| 172.22.1.250 | | Arista 100G | |
| IPv4 | IPv6 | DEVICE | dhcpd host | Helpful URLs |
| ---------------------- | ---- | --------------------------------------------- | ------------ | ------------------------------------------------------- |
| public ip / 172.22.0.1 | | gateway/mgmt | (static) | <http://172.22.0.1:9091/> and <http://172.22.0.1:3000/> |
| 172.22.0.5 | | ToR, Arista DCS-7280TR-48C6-R | tor | |
| 172.22.0.10 | | Network PDU | npdu | |
| 172.22.0.20 | | KVM | kvm | |
| 172.22.4.104 | | TS, Vertiv Avocent ACS8032, see note 5 | ts | <https://172.22.4.104/> |
| 172.22.0.227 | | F5 BIG-IP-01 (F5A), internal | (static) | |
| 172.22.0.228 | | F5 BIG-IP-02 (F5B), internal | (static) | |
| 172.22.0.229 | | F5 BIG-IP floating, internal | (static) | |
| | | | | |
| 172.22.1.1 | | DPU Host 1 | dh1 | |
| 172.22.2.1 | | DH1 iDRAC | dh1idrac | <https://172.22.2.1/redfish/v1/> |
| 172.22.3.1 | | AMD DSC200 | (none yet) | |
| | | | | |
| 172.22.1.2 | | DPU Host 2 | dh2 | |
| 172.22.2.2 | | DH2 iLO | dh2ilo | <https://172.22.2.2/redfish/v1/> |
| 172.22.3.2 | | Nvidia BF2 | dh2bf2 | |
| 172.22.4.2 | | Nvidia BF2 bmc | dh2bf2bmc | <https://172.22.4.2/redfish/v1/> |
| | | | | |
| 172.22.1.3 | | DPU Host 3 | dh3 | |
| 172.22.2.3 | | DH3 iLO | dh3ilo | <https://172.22.2.3/redfish/v1/> |
| 172.22.3.3 | | Marvell CN106 (reserved, see note 1) | (none) | |
| 172.22.4.3 | | Marvell CN106 MCU (Zephyr) | dh3cn106mcu | |
| | | | | |
| 172.22.1.4 | | DPU Host 4 | dh4 | |
| 172.22.2.4 | | DH4 iDRAC | dh4idrac | <https://172.22.2.4/redfish/v1/> |
| 172.22.4.4 | | Intel IPU E2100 (MacKay Peak) IMC, see note 2 | dh4e2100imc | |
| | | | | |
| 172.22.1.5 | | DPU Host 5 | dh5 | |
| 172.22.2.5 | | DH5 iDRAC | dh5idrac | <https://172.22.2.5/redfish/v1/> |
| 172.22.3.5 | | Nvidia BF3, see note 3 | dh5bf3 | |
| 172.22.4.5 | | Nvidia BF3 bmc | dh5bf3bmc | <https://172.22.4.5/redfish/v1/> |
| | | | | |
| 172.22.1.9 | | DPU Host 9 | dh9 | |
| 172.22.2.9 | | DH9 BMC, dedicated port | dh9bmc | <https://172.22.2.9/redfish/v1/> |
| 172.22.2.109 | | DH9 BMC, shared host NIC (NC-SI) | dh9bmcshared | |
| 172.22.3.9 | | Intel Mount Evans (DYP), see note 4 | dh9dpu | |
| 172.22.4.9 | | Intel Mount Evans on-card BMC | dh9dpubmc | |
| | | | | |
| 172.22.1.100 | | Tgen1 | tgen1 | |
| 172.22.2.100 | | Tgen1 bmc | tgen1bmc | <https://172.22.2.100/redfish/v1/> |
| | | | | |
| 172.22.1.250 | | Arista 100G | arista100g | |

Notes:

1. CN106 Linux has no OOB link: console on DH3 `/dev/ttyUSB0`, MCU on `/dev/ttyUSB1`.
2. ACC reached through the IMC at 192.168.0.2 (`mev` in the ansible inventory).
3. BF3 Arm and BMC share one RJ45.
4. Moved from DH4; not yet verified.
5. TS is static at 172.22.4.104, not its dhcpd address 172.22.0.30; consoles in [hardware/TS](../hardware/TS/README.md).

F5 management: 192.168.0.227 (F5A) and 192.168.0.228 (F5B), reached from mgmt.
10 changes: 9 additions & 1 deletion ansible/inventory
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# from https://github.com/opiproject/opi-poc/blob/main/lab/ips.md
# keep in sync with ../Documentation/ips.md and ../hardware/mgmt/fs/etc/dhcp/dhcpd.conf

[hostservers]
dh1 ansible_host=172.22.1.1 ansible_connection=ssh ansible_user=root
Expand All @@ -18,15 +18,23 @@ dh9bmc ansible_host=172.22.2.9 ansible_connection=local ansible_user=root
[DPUs]
amd ansible_host=172.22.3.1 ansible_connection=ssh ansible_user=root
bf2 ansible_host=172.22.3.2 ansible_connection=ssh ansible_user=root
# CN106 in DH3: no OOB link, console only
mrv ansible_host=172.22.3.3 ansible_connection=ssh ansible_user=root
# DH4 Intel E2100; name mev kept for telegraf and Grafana
mev ansible_host=192.168.0.2 ansible_connection=ssh ansible_user=root ansible_ssh_common_args='-J root@172.22.4.4' proxy_env="{'http_proxy': 'http://192.168.0.1:3128', 'https_proxy': 'http://192.168.0.1:3128'}"
bf3 ansible_host=172.22.3.5 ansible_connection=ssh ansible_user=root
# DH9 Mount Evans, credentials unknown
#dh9dpu ansible_host=172.22.3.9 ansible_connection=ssh ansible_user=root

[dpubmcs]
bf2bmc ansible_host=172.22.4.2 ansible_connection=local ansible_user=root
# CN106 MCU (Zephyr shell, no ssh)
mrvbmc ansible_host=172.22.4.3 ansible_connection=local ansible_user=root
# E2100 IMC in DH4
mevbmc ansible_host=172.22.4.4 ansible_connection=ssh ansible_user=root
bf3bmc ansible_host=172.22.4.5 ansible_connection=local ansible_user=root
# DH9 Mount Evans BMC, credentials unknown
#dh9dpubmc ansible_host=172.22.4.9 ansible_connection=local ansible_user=root

[tgens]
tgen1 ansible_host=172.22.1.100 ansible_connection=ssh ansible_user=root
Expand Down
24 changes: 17 additions & 7 deletions hardware/TS/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,36 @@
# Serial Consoles

IP to serial switch for remote access to the devices serial port
ACS8000 48-port unit dual AC power supply
ACS8032 32-port unit, internal modem, dual AC power supply

## HW spec

- Part Number: ACS8048DAC-400
- Firmware: 2.26.1.5874+902+37+17 (Nov 7 2023 - 11:53:23)
- Model: ACS8032 with internal modem and dual power supply
- Firmware: 2.30.2.6525+920+41+18 (Jul 9 2025 - 20:45:03)

## Management

- 1G ethernet management port
- 1G ethernet management port, ToR Et3, 172.22.4.104 (static; dhcpd reserves 172.22.0.30)
- Security profile "Secure": SSH and HTTPS only, no telnet, no ping
- Serial port N: `ssh -p $((8000+N)) admin@172.22.4.104`, hot key `^Z`

## Docs

[User guide](https://www.vertiv.com/49db15/globalassets/shared/avocent-acs-8000-installer-userguide.pdf)

## Devices connected

- Network PDU
- 100G Switch
- KVM
Checked 2026-10-19 by opening each port.

| Port | Name | Device | Speed |
|------|-------------|--------------------------------|--------|
| 1 | Network-PDU | Vertiv Geist rack PDU | 115200 |
| 2 | A100G | Arista DCS-7280CR-48-F (A100G) | 9600 |
| 3 | | unknown, no output | 19200 |
| 4 | BIG-IP-02 | F5B, BIG-IP i4000 | 19200 |
| 5 | ToR | Arista DCS-7280TR-48C6-R | 9600 |
| 6 | | unknown, no output at any rate | 19200 |
| 7 | BIG-IP-01 | F5A, BIG-IP i4000 | 19200 |

## Pictures

Expand Down
56 changes: 44 additions & 12 deletions hardware/ToR/arista.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,42 @@
! Command: show running-config
! device: tor (DCS-7280TR-48C6, EOS-4.22.3M)
! device: tor (DCS-7280TR-48C6, EOS-4.32.0F)
!
! boot system flash:/EOS-4.22.3M.swi
! boot system flash:/EOS-4.32.0F.swi
!
enable password sha512 <hash>
no aaa root
!
username admin privilege 15 role network-admin secret sha512 <hash>
username arista secret sha512 <hash>
!
dhcp server
!
transceiver qsfp default-mode 4x10G
!
service routing protocols model multi-agent
!
hostname tor
!
spanning-tree mode mstp
!
enable password sha512 <hash>
no aaa root
system l1
unsupported speed action error
unsupported error-correction action error
!
username admin privilege 15 role network-admin secret sha512 <hash>
username arista secret sha512 <hash>
vlan 10
name public-he-net
!
vlan 100,102,200
!
management api gnmi
transport grpc openmgmt
port 5900
provider eos-native
!
interface Ethernet1
speed forced 1000full
switchport access vlan 10
ip access-group PUBLIC-IN in
!
interface Ethernet2
speed forced 1000full
Expand All @@ -34,9 +52,11 @@ interface Ethernet5
!
interface Ethernet6
speed forced 1000full
switchport access vlan 10
!
interface Ethernet7
speed forced 1000full
switchport access vlan 10
!
interface Ethernet8
speed forced 1000full
Expand Down Expand Up @@ -85,9 +105,12 @@ interface Ethernet22
!
interface Ethernet23
speed forced 1000full
switchport access vlan 10
!
interface Ethernet24
speed forced 1000full
switchport trunk allowed vlan 1,100,102,200
switchport mode trunk
!
interface Ethernet25
speed forced 1000full
Expand All @@ -109,9 +132,12 @@ interface Ethernet30
!
interface Ethernet31
speed forced 1000full
switchport access vlan 10
!
interface Ethernet32
speed forced 1000full
switchport trunk allowed vlan 1,100,102,200
switchport mode trunk
!
interface Ethernet33
speed forced 1000full
Expand Down Expand Up @@ -177,19 +203,25 @@ interface Ethernet53/1
speed forced 100gfull
!
interface Ethernet54/1
shutdown
speed forced 100gfull
switchport access vlan 102
!
interface Management1
ip address 172.22.0.5/16
!
ip route 0.0.0.0/0 172.22.0.1
interface Vlan102
!
ip access-list PUBLIC-IN
10 permit tcp any host 216.218.200.249 eq https
20 permit tcp any host 216.218.200.249 established
30 permit udp any eq ntp host 216.218.200.249
40 permit udp any eq domain host 216.218.200.249
50 permit icmp any host 216.218.200.249
60 deny ip any host 216.218.200.249
1000 permit ip any any
!
no ip routing
!
management api gnmi
transport grpc openmgmt
port 5900
provider eos-native
ip route 0.0.0.0/0 172.22.0.1
!
end
7 changes: 7 additions & 0 deletions hardware/dh4/fs/etc/netplan/60-opilab-dh4-port2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# DH4 port 2 (Et30): no DHCP, a second lab /16 address causes ARP flux
network:
version: 2
ethernets:
eno12409np1:
dhcp4: false
dhcp6: false
18 changes: 18 additions & 0 deletions hardware/mgmt/fs/etc/default/isc-dhcp-server
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Defaults for isc-dhcp-server (sourced by /etc/init.d/isc-dhcp-server)

# Path to dhcpd's config file (default: /etc/dhcp/dhcpd.conf).
#DHCPDv4_CONF=/etc/dhcp/dhcpd.conf
#DHCPDv6_CONF=/etc/dhcp/dhcpd6.conf

# Path to dhcpd's PID file (default: /var/run/dhcpd.pid).
#DHCPDv4_PID=/var/run/dhcpd.pid
#DHCPDv6_PID=/var/run/dhcpd6.pid

# Additional options to start dhcpd with.
# Don't use options -cf or -pf here; use DHCPD_CONF/ DHCPD_PID instead
#OPTIONS=""

# On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
# Separate multiple interfaces with spaces, e.g. "eth0 eth1".
INTERFACESv4="enp1s0f0np0"
INTERFACESv6=""
Loading
Loading