Skip to content

OPI lab remap: DHCP reservations, ToR public VLAN and mgmt hardening - #114

Open
mgheorghe wants to merge 20 commits into
opiproject:mainfrom
mgheorghe:opi-lab-remap
Open

mgheorghe wants to merge 20 commits into
opiproject:mainfrom
mgheorghe:opi-lab-remap

Conversation

@mgheorghe

@mgheorghe mgheorghe commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Inventory and remap of the OPI test rack, verified on 2026-10-19 against the live lab (ToR MAC table, ARP sweep from mgmt, and logins to each device).

DHCP (hardware/mgmt/fs/etc/dhcp/dhcpd.conf)

  • Synced with the live config on mgmt; fixed the ToR reservation and removed the duplicate subnet.
  • New reservations: BlueField-3 DPU and BMC (172.22.3.5 / 172.22.4.5), terminal server (172.22.0.30), CN106 MCU (172.22.4.3), DH9 DPU BMC (172.22.4.9), DH9 shared-NIC BMC (172.22.2.109).
  • DH4 card corrected to the Intel IPU E2100 MacKay Peak; the Mount Evans card moved to DH9.
  • isc-dhcp-server is bound to the lab NIC only.

DH4: the second onboard port no longer takes a DHCP lease (it caused ARP flux on the shared /16).

ToR (hardware/ToR/arista.config)

  • Synced with the live running-config (EOS 4.32.0F).
  • New VLAN 10 for the he.net uplink, the mgmt public NIC, the mgmt BMC and the BIG-IP external ports. Lab VLAN 1 no longer shares an L2 domain with the internet.
  • BIG-IP trunks pinned to VLANs 1,100,102,200.
  • Ingress ACL on Et1 limits internet access to the mgmt BMC.

mgmt: fail2ban sshd jail.

All live changes were applied with an EOS commit timer and verified before being confirmed.

Docs and inventory

  • Documentation/ips.md and ansible/inventory rewritten from the sweep; the TS answers on 172.22.4.104.
  • Rack diagram (images/opi-lab-cabling.drawio.svg): DH9 moved to U37-40, DH4 relabelled E2100, DH9 Mount Evans added, wrong IPs fixed, TS console cables redrawn.
  • hardware/TS/README.md: ACS8032 model, firmware, SSH access and the console port table.

Supersedes #103.

Still to come on this branch: ToR port map in Documentation/cables.md, and deployment of the final dhcpd.conf.

Closes #59: the drawing shows mgmt and dh9 with their connections.

Copy /etc/dhcp/dhcpd.conf from 172.22.0.1 verbatim (2026-10-19) so later
changes are reviewable diffs against what is actually running.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
The tor entry used the VLAN 102 CPU MAC (:65) and 172.22.1.250, which is
already reserved for arista100g. Point it at the ToR's Management1 MAC
28:99:3a:9c:39:64 and its real address 172.22.0.5.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
The empty 172.22.0.0/16 block was a leftover from the ISC sample file and
duplicated the real subnet that carries the 172.22.222.0/24 pool.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
BF3 in DH5 (ToR Et36) had no reservation and was leasing 172.22.222.2/.3
from the unknown pool, so 172.22.3.5 and 172.22.4.5 looked down. Pin the
Arm OOB (:f4) to 172.22.3.5 and the on-card BMC (:f5) to 172.22.4.5.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
TBD02 (Senao OUI d4:e5:c9:00:ec:30, ToR Et34) is the OpenBMC on the DH9
DPU, next to dh9dpu (:31). Name it dh9dpubmc and give it the DPU-BMC
address matching its host number.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
TBD03 (a0:36:bc:cb:cf:f8) is DH9's AMI BMC answering through the shared
host NIC on ToR Et28 (dedicated port is dh9bmc on Et29). Move it out of
the DPU-BMC range into the BMC range as dh9bmcshared.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
TBD04 (Avocent OUI 00:e0:86:37:59:9c, ToR Et3 = TS port on the cabling
diagram) is the ACS8000. Give it its planned 172.22.0.30 address and drop
the old commented ts entry whose MAC was never seen on the ToR.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
TBD01 (02:04:9f:70:8e:be, ToR Et2) is the CN106 management MCU (Zephyr,
'net iface' on DH3 /dev/ttyUSB1), not the DPU OS. Replace the placeholder
MAC in dh3cn106mcu with it and drop TBD01. The CN106 Linux has no OOB
link and is reached on DH3 /dev/ttyUSB0.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
The DH4 IPU E2100 IMC now presents 10:2e:00:00:fe:fe and was leasing
172.22.222.4, which broke the ansible jump host (root@172.22.4.4) for mev.
Point the reservation at the new MAC and record the MAC history.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
eno12409np1 (d4:04:e6:2a:b0:21, ToR Et30) was leasing 172.22.222.1 on the
same /16 as eno12399np0, giving two default routes and ARP flux. Add a
netplan override (sorts after 50-cloud-init.yaml so cloud-init cannot undo
it). Applied on DH4 2026-10-19; netplan apply alone kept the old lease,
'networkctl reconfigure eno12409np1' removed it.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Correct the dh4 IMC comment from 00715dc: DH4 carries an Intel IPU
Adapter E2100-CCQDA2HL (MacKay Peak), and the Mount Evans (DYP) card was
moved to DH9, so the old MACs predate a card swap. Rename dh4dypimc to
dh4e2100imc and label the DH9 DPU entries.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
The repo copy predated the EOS 4.32.0F upgrade and was missing VLANs
100/102/200, the Et24/Et32 trunks to the BIG-IP pair and the Et54/1
VLAN 102 link to the A100G.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Et1 (he.net uplink), Et6 (mgmt public NIC), Et7 (mgmt BMC) and Et23/Et31
(BIG-IP external) move to VLAN 10. Lab VLAN 1 no longer shares an L2
domain with the internet. The BIG-IP trunks Et24/Et32 are pinned to
VLANs 1,100,102,200 so VLAN 10 never reaches them.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
INTERFACESv4 was empty, so dhcpd listened on every interface including
the public one. Restrict it to enp1s0f0np0 (172.22.0.1/16).

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Ingress ACL PUBLIC-IN on the he.net uplink: from the internet the mgmt
BMC (216.218.200.249) only accepts HTTPS, replies to its own sessions,
NTP/DNS answers and ICMP. SSH and IPMI (623) from the internet are
dropped. Lab and VPN users still reach it through mgmt on VLAN 10,
which does not cross Et1.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
mgmt sshd on the public IP saw ~17k failed root logins a day. Ban for
1h after 5 failures in 10 min; lab, WireGuard, BIG-IP management and
the public /27 are never banned.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
…0-19 sweep

Fix the F5 internal addresses (.227 F5A, .228 F5B, .229 floating), add
the dhcpd host names, the CN106 MCU, the DH4 E2100 IMC, the DH9 BMC
(dedicated and shared) and the Mount Evans card now in DH9. Keep the
mev/mevbmc inventory names, which telegraf and Grafana depend on, and
add the DH9 DPU commented out until its credentials are known.

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
…0-19 sweep

- move DH9 to the bottom group (U37-40, above DH5)
- relabel DH4 DPU as Intel IPU E2100 (MacKay Peak), IMC 172.22.4.4
- add DH9 Mount Evans 172.22.3.9 / BMC 172.22.4.9, fix DH5 title and host IP
- fix F5 model typo, add BIG-IP-01/02 names; drop the bogus mgmt IPMI 172.22.2.0
- rewire TS consoles: 1 PDU, 2 A100G, 4 F5B, 5 ToR, 7 F5A; label TS 172.22.4.104
- hardware/TS: ACS8032 model, firmware, SSH access and the port table
- ips.md: TS answers on 172.22.4.104

Signed-off-by: Mircea Dan Gheorghe <mircea-dan.gheorghe@keysight.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Rack diagram doesnot show the Management server and dh9 server and their connections

1 participant