Appendix B - Reference configuration¶

This appendix is the reference configuration set of the platform in one place: the TRex configuration files, the systemd units and the host protection, the complete configurations of both TX switches, the attachment-point ports on the Leafs and Border Leafs, the requirements for the firewall test zone (the default option B and the alternative option A), the planned trexctl inventory and the default monitoring profile, and the InfluxDB and Grafana set-up. Every block carries a status badge and its source. Values in angle brackets (<MGMT-IP-TX-K>) are placeholders; interface numbers of the TX switches depend on the module layout of the spare chassis and must be confirmed on the device; the trexctl files show a planned format - the tool does not exist yet, and Appendix A.4 shows how the same is done with native TRex. The reasoning behind each block is in the chapters linked from its section.

How to read the Comware blocks

Lines that start with # are comments (HPE documentation convention, Fundamentals Configuration Guide R630x, PDF p. 209) - do not type them. Optional commands are shown as such comments and explained in the table below the block. All Comware commands use syntax that is either present in the LVR running configurations of February 2025 or documented in the HPE FlexFabric 5950 R630x guides; the page references are PDF pages.

B.1 TRex configuration files¶

Two TRex instances share trex-rzk-01: trex-mon on NIC1 and trex-lab on NIC2. trex-lab runs either stateless (trex_lab.yaml, four ports) or in ASTF mode with its own two-port configuration (trex_lab_astf.yaml), never both at the same time. The instances can share the server because each has its own NIC, its own DPDK file prefix, its own ZMQ ports, its own memory limit and its own CPU threads (TRex manual, section 6.2 "Platform YAML (--cfg argument)"). The PCI addresses are placeholders: take them from ./dpdk_setup_ports.py -s (NIC1 = the four functions .0-.3 of the first X710, NIC2 = the second X710). Explained in Chapter 3 and Chapter 17.

verified configuration keys (TRex manual 6.2.1 and 6.2.3) and queue-mode selection (src/trex_modes.h, choose_mode() in src/trex_modes.cpp) · confirm on device PCI addresses and thread numbers · source: plan v0.92 section 8.5; thread layout and queue mode as in Chapter 17

/etc/trex/trex_mon.yaml - monitoring instance (NIC1, software mode, EMU)

# Example thread layout, 1 CPU with 16 cores / 32 threads, siblings n and n+16 (check: lscpu -e):
#   trex-mon : master 4, latency (RX) 5, data plane 6 (p0/p1) and 8 (p2/p3)
#   trex-lab : master 10, latency (RX) 11, data plane 12-15 and 28-31
#   OS, InfluxDB, Grafana and trexctl: threads 0-3 and 16-19 (physical cores 0-3)
#   kernel command line: isolcpus=4-15,20-31 keeps the TRex threads and their siblings free
- version: 2
  interfaces: ['31:00.0', '31:00.1', '31:00.2', '31:00.3']   # (1)!
  port_limit: 4
  prefix: mon                    # (2)!
  zmq_pub_port: 4500
  zmq_rpc_port: 4501
  limit_memory: 3072             # (3)!
  port_bandwidth_gb: 10
  c: 1                           # (4)!
  services_core: 4               # (5)!
  port_info:                     # (6)!
    - ip: 1.1.1.1
      default_gw: 1.1.1.2
    - ip: 1.1.1.2
      default_gw: 1.1.1.1
    - ip: 1.1.2.1
      default_gw: 1.1.2.2
    - ip: 1.1.2.2
      default_gw: 1.1.2.1
  platform:
    master_thread_id: 4
    latency_thread_id: 5
    dual_if:
      - socket: 0
        threads: [6]
      - socket: 0
        threads: [8]

The order of this list defines the TRex port numbers: first entry = port 0 = p0.
prefix, the two ZMQ ports, limit_memory and the threads must differ between the two instances; the prefix is passed to DPDK as --file-prefix.
Megabytes of packet memory (passed to DPDK as -m). Both instances together (6 GB) must fit into the 8 GB of hugepages reserved on the kernel command line.
One data-plane thread per port pair. In stateless software mode c: 1 selects the queue mode ONE_QUE: the RX core reads the receive queue of each port itself, so latency is measured accurately. With c of 2 or more TRex selects MULTI_QUE, in which the data-plane threads read the queues and pass the latency packets on - "latency won't be accurate in this mode" (src/trex_modes.h). About 20.6 kpps need far less than one thread. After every start grep 'queues mode' /var/log/trex/trex-mon.log must show set dpdk queues mode to ONE_QUE. Every threads list needs at least c entries; the thread of the idle pair p2/p3 (8) still polls at 100 %.
CPU on which the helper services (Scapy server, TRex-EMU) are pinned with taskset. When the key is absent TRex uses master_thread_id (manual 6.2.1; get_core_for_services() in dpdk_setup_ports.py), and the helper script advises "free core or TRex master thread core" (scripts/general_daemon_server). The master thread 4 is used, not the RX thread 5, which busy-polls as long as latency streams are active (hot_state_loop() in src/stx/common/trex_rx_core.cpp). confirm on device on the bench.
Placeholders written by dpdk_setup_ports.py -i. In stateless mode the platform does not use them: EMU provides the monitoring client addresses, the test tooling the lab addresses. The src_mac entries of the lab ports in trex_lab.yaml are different: they are real and fixed (see below).

/etc/trex/trex_lab.yaml - lab instance, stateless (NIC2, hardware mode)

- version: 2
  interfaces: ['98:00.0', '98:00.1', '98:00.2', '98:00.3']   # p4-p7 = ports 0-3
  port_limit: 4
  prefix: lab
  zmq_pub_port: 4600
  zmq_rpc_port: 4601
  limit_memory: 3072
  port_bandwidth_gb: 10
  c: 4
  port_info:                     # (1)!
    - src_mac: '02:54:52:f0:00:04'
      ip: 2.1.1.1
      default_gw: 2.1.1.2
    - src_mac: '02:54:52:f0:00:05'
      ip: 2.1.1.2
      default_gw: 2.1.1.1
    - src_mac: '02:54:52:f0:00:06'
      ip: 2.1.2.1
      default_gw: 2.1.2.2
    - src_mac: '02:54:52:f0:00:07'
      ip: 2.1.2.2
      default_gw: 2.1.2.1
  platform:
    master_thread_id: 10
    latency_thread_id: 11
    dual_if:
      - socket: 0
        threads: [12, 13, 28, 29]
      - socket: 0
        threads: [14, 15, 30, 31]

The src_mac of each lab port is its fixed identity (p4 02:54:52:f0:00:04 ... p7 02:54:52:f0:00:07, in the order of the interfaces list): TRex answers ARP for a lab port with the port MAC and runs ASTF with it, and a lab port is re-pinned to other APs over time, so the MAC must not depend on the AP. Only the lab IP follows the pinned AP (lab pool start .151 + AP index - 1) - the ip and default_gw values here are placeholders, the test runner sets the real ones per pin with set_l3_mode. The same MACs are recorded as mac: in the inventory ports.lab (B.7) and protected by the static MAC entries of B.3.

In hardware mode the X710 steers the latency packets by a filter directly to the RX core (queue mode DROP_QUE_FILTER), so the measurement stays accurate with c: 4.

ASTF assigns fixed roles: even TRex ports are the client side, odd ports the server side (TRex ASTF guide). In trex_lab.yaml p6 would be port 2 - a client port. The ASTF unit therefore uses its own configuration that lists only p4 and p6, in this order; the planned trexctl test astf rewrites it for the chosen pins (Chapter 7, Chapter 22).

sketch - validate on the bench · decision zone address and VLAN (firewall-zone option B shown, B.6)

/etc/trex/trex_lab_astf.yaml - lab instance in ASTF mode (p4 client side, p6 server side)

- version: 2
  interfaces: ['98:00.0', '98:00.2']   # (1)!
  port_limit: 2
  prefix: lab
  zmq_pub_port: 4600
  zmq_rpc_port: 4601
  limit_memory: 3072
  port_bandwidth_gb: 10
  c: 4
  port_info:
    - src_mac: '02:54:52:f0:00:04'   # (2)!
      ip: 10.3.14.151
      default_gw: 10.3.14.1
      vlan: 1314
    - src_mac: '02:54:52:f0:00:06'   # (3)!
      ip: 10.9.90.156
      default_gw: 10.9.90.1
      vlan: 4003
  platform:
    master_thread_id: 10
    latency_thread_id: 11
    dual_if:
      - socket: 0
        threads: [12, 13, 28, 29]

p4 (.0) becomes port 0 (client side), p6 (.2) port 1 (server side); p5 and p7 stay unused while the ASTF unit runs. Prefix, ZMQ ports and memory are those of trex-lab: the two lab units never run together. Because the file uses only two of the four ports of the second X710, the unit starts TRex with --unbind-unused-ports (B.2); without it TRex exits with an error about bound but unused ports of the same NIC confirm on device.
Client side: p4's fixed port MAC and the lab address of AP K-L11 in VLAN 1314 (lab pool start .151 + index 1 - 1), gateway Vsi-interface914 on sw-rzk-BL. p4 is pinned to K-L11 and sends single-tagged frames; vlan also tags the port's own ARP. The lab address needs its own IPAM reservation and DHCP exclusion.
Server side: p6's fixed port MAC and the lab address in the firewall zone (a TRex-only subnet), gateway = zone VIP 10.9.90.1. p6 is pinned to FWZ, the zone VLAN 4003 as reached through AP K-BL (index 6): lab IP .151 + 6 - 1 = 10.9.90.156. The TX switch adds S-VLAN 3110 (PVID of p6), TRex sends VLAN 4003. With firewall-zone option A the zone is AP FW (index 14): 10.9.90.164, vlan: 1420, p6 PVID 3300 - the src_mac stays 02:54:52:f0:00:06.

Key	`trex-mon`	`trex-lab`	`trex-lab-astf`	Why different
`interfaces`	NIC1 (p0-p3)	NIC2 (p4-p7)	p4, p6	a NIC belongs to exactly one instance; ASTF needs the server side on an odd port
`prefix`	`mon`	`lab`	`lab`	separate DPDK runtime files and hugepage files; the lab units share them because they never run together
`zmq_pub_port` / `zmq_rpc_port`	4500 / 4501	4600 / 4601	4600 / 4601	console and API must reach the right server
`c` and `dual_if` threads	1 per port pair: [6], [8]	4 per port pair	4, one pair	`ONE_QUE` for accurate latency in software mode; line-rate tests need more TX capacity
`services_core`	4 (= master thread)	not set (= master thread 10)	not set (= master thread 10)	keep EMU away from the busy-polling RX thread

Check the hyper-thread siblings

The layout assumes Linux numbering in which CPUs n and n+16 are the two threads of one physical core. With this numbering the OS, InfluxDB and Grafana run on threads 0-3 and 16-19 (physical cores 0-3), and the kernel parameter isolcpus=4-15,20-31 keeps the TRex threads and their siblings free of other work (Chapter 17). If lscpu -e=CPU,CORE shows a different numbering, adapt the thread numbers and the isolcpus list confirm on device.

B.2 systemd units¶

verified server flags (TRex manual 6.3, src/main_dpdk.cpp) · confirm on device paths and the --unbind-unused-ports behaviour · source: plan v0.92 section 8.6

/etc/systemd/system/trex-mon.service

[Unit]
Description=TRex monitoring instance (stateless, software mode, EMU)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/trex/current
ExecStart=/opt/trex/current/t-rex-64 -i --software --emu --cfg /etc/trex/trex_mon.yaml --iom 0
Restart=on-failure
RestartSec=15
LimitMEMLOCK=infinity
LimitNOFILE=65536
StandardOutput=append:/var/log/trex/trex-mon.log
StandardError=append:/var/log/trex/trex-mon.log

[Install]
WantedBy=multi-user.target

/etc/systemd/system/trex-lab.service

[Unit]
Description=TRex lab instance (stateless, hardware flow statistics)
After=network-online.target
Conflicts=trex-lab-astf.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/trex/current
ExecStart=/opt/trex/current/t-rex-64 -i --cfg /etc/trex/trex_lab.yaml --iom 0 --no-scapy-server
Restart=no
LimitMEMLOCK=infinity
StandardOutput=append:/var/log/trex/trex-lab.log
StandardError=append:/var/log/trex/trex-lab.log

/etc/systemd/system/trex-lab-astf.service

[Unit]
Description=TRex lab instance (advanced stateful, ASTF)
After=network-online.target
Conflicts=trex-lab.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/trex/current
ExecStart=/opt/trex/current/t-rex-64 -i --astf --cfg /etc/trex/trex_lab_astf.yaml --iom 0 --no-scapy-server --unbind-unused-ports
Restart=no
LimitMEMLOCK=infinity
StandardOutput=append:/var/log/trex/trex-lab.log
StandardError=append:/var/log/trex/trex-lab.log

Activation

mkdir -p /etc/trex /var/log/trex
systemctl daemon-reload
systemctl enable --now trex-mon.service
tail -n 30 /var/log/trex/trex-mon.log
grep 'queues mode' /var/log/trex/trex-mon.log    # set dpdk queues mode to ONE_QUE
systemctl start trex-lab.service    # on demand (Conflicts= with trex-lab-astf)

Element	Why
`WorkingDirectory=/opt/trex/current`	`t-rex-64` calls `./trex-cfg` relative to the current directory
`-i ... --iom 0`	interactive server without the text UI in the log
`Conflicts=` between the lab units	only one of them may own NIC2
`--cfg /etc/trex/trex_lab_astf.yaml` in the ASTF unit	ASTF needs the server side on an odd port: p4 = port 0 (client), p6 = port 1 (server)
`--no-scapy-server` in both lab units	the Scapy server (port 4507) is one named helper daemon per host; every interactive stateless start restarts it, unless this flag is given. With the flag, starting or stopping a lab unit leaves the helper of `trex-mon` alone (next block)
`--unbind-unused-ports` in the ASTF unit	the ASTF file uses two of the four ports of the second X710; without the option TRex (i40e) exits with an error about bound but unused ports of the same NIC confirm on device
no `--emu` in the lab units	the EMU server (port 4510) is also one named daemon per host: a second `--emu` start would restart the EMU server of `trex-mon`
no `[Install]` in the lab units	they are started on demand, never at boot
`StandardOutput=append:`	TRex output goes to `/var/log/trex/*.log`; `journalctl -u` shows only unit events
`Restart=on-failure` only for `trex-mon`	the monitoring comes back by itself; the exporter's supervisor thread notices the restart (server uptime reset, or `active_streams` at zero while a profile is recorded as active in `/data/trexctl/state/`) and runs `trexctl apply` for the active profile once the server accepts connections again

Side effects of every TRex start

t-rex-64 runs trex-cfg and dpdk_setup_ports.py before the server starts. Read from the trex-core source:

trex-cfg runs systemctl stop firewalld.service on Red Hat-family systems (scripts/trex-cfg), at every call of the wrapper. The TRex control ports (RPC 4501/4601, publisher 4500/4600, EMU 4510, Scapy server 4507) listen on all addresses and have no authentication. Host protection is therefore a design decision - see the next block.
The helper services are global, named daemons with fixed default ports: Scapy server 4507 and EMU 4510. An instance starts them with general_daemon_server restart -n <name> and stops them by name when it exits - the Scapy server only for an interactive stateless start without --no-scapy-server (ASTF starts skip it unless --scapy-server is given), the EMU server only with --emu (should_scapy_server_run(), run_servers() and cleanup_servers() in scripts/dpdk_setup_ports.py; scripts/general_daemon_server). Without the flag, starting or stopping trex-lab would restart or stop the Scapy server that trex-mon started - hence --no-scapy-server on both lab units (the ASTF unit carries it as well, which makes the rule explicit). The console and the Python API do not use the Scapy server (only the TRex GUI does). The lab units never use --emu: the restart would hit the EMU server of trex-mon. confirm on device on the bench (Appendix D).

Open decision: host protection - nftables allow-list instead of firewalld

Because every TRex start stops firewalld, a firewalld-based host firewall cannot protect trex-rzk-01. The plan (section 8.3) proposes the native nftables service, which trex-cfg does not touch, with firewalld disabled and removed, plus ACLs on the management network: only the management hosts reach SSH, HTTPS (Grafana) and the six TRex control ports 4500, 4501, 4507, 4510, 4600 and 4601 (for consoles and scripts on an administrator's workstation); InfluxDB listens on 127.0.0.1 only, and the local clients (exporter, trexctl, consoles on the server) use the loopback interface. A stricter variant leaves out the TRex-port line, so that the control ports are reachable from the server itself only. Alternatives: restart firewalld after every TRex start (a short unprotected window at every start), or patch trex-cfg (repeat after every TRex upgrade). Owner: LVR Linux team with the platform administrator. See Chapter 17 and Appendix D.

decision · confirm on the target OS (RHEL 9 nftables.service loads /etc/sysconfig/nftables.conf; test from the iLO console, because the policy is drop)

Switch from firewalld to nftables - management NIC of trex-rzk-01

systemctl disable --now firewalld
dnf -y remove firewalld            # or: systemctl mask firewalld, if the package has to stay
cat >/etc/sysconfig/nftables.conf <<'EOF'
flush ruleset
table inet filter {
    set mgmt_hosts {
        type ipv4_addr; flags interval
        elements = { <MGMT-PREFIX>, <JUMPHOST-IP> }
    }
    chain input {
        type filter hook input priority 0; policy drop;
        iif lo accept
        ct state established,related accept
        ct state invalid drop
        ip protocol icmp accept
        # meta l4proto ipv6-icmp accept   # only if the management network uses IPv6
        ip saddr @mgmt_hosts tcp dport { 22, 443 } accept
        # TRex control channels (stricter variant: leave this line out)
        ip saddr @mgmt_hosts tcp dport { 4500, 4501, 4507, 4510, 4600, 4601 } accept
        counter drop
    }
    chain forward { type filter hook forward priority 0; policy drop; }
    chain output  { type filter hook output priority 0; policy accept; }
}
EOF
systemctl enable --now nftables
nft list ruleset
systemctl restart trex-mon && systemctl is-active nftables   # still active after a TRex start

The DPDK ports do not pass through the kernel, so the ruleset only concerns the management NIC. Acceptance test: from a host outside <MGMT-PREFIX>, SSH, HTTPS and TCP 4500-4510 / 4600-4601 are unreachable - also after a restart of trex-mon; from a management host, SSH and HTTPS work.

B.3 TX switch sw-rzk-TX1¶

Assumed hardware: spare HPE FlexFabric 5950 4-slot (JH404A) with a 24 x SFP+ module in slot 1 (Ten-GigabitEthernet1/1/x) and a QSFP28 module in slot 2 (HundredGigE1/2/x), factory-reset, release R6301P03 like the Border Leafs. Check the module layout with display device manuinfo (Fundamentals Command Reference, PDF p. 273) before you adapt the interface names. Explained in Chapter 12.

The configuration below is the default with firewall-zone option B: the zone is reached through AP K-BL, so no firewall is cabled to the TX switch. The option-A firewall port XGE1/1/7 (CPF-ZV01) is nevertheless configured (access, PVID 3300, QinQ) but kept shutdown, and S-VLAN 3300 is defined and permitted on the trunks without an active member port - harmless with option B, so that option A later needs only the firewall cabling and undo shutdown. What option A adds follows after the explanation table decision.

verified syntax (5950 R630x guides, LVR running configurations) · confirm on device interface numbers, management address, remote ports · source: plan v0.92 sections 6.1 and 7.1

sw-rzk-TX1 - complete configuration (firewall-zone option B, the default)

system-view
 sysname sw-rzk-TX1
 clock timezone COLOGNE add 01:00:00
 clock summer-time MEST 02:00:00 March last Sunday 02:00:00 October last Sunday 01:00:00
 clock protocol ntp
#
# --- management (same pattern as the production switches) ---
ip vpn-instance MGMT
 route-distinguisher 10:1
 description MGMT/Monitoring
 vpn-target 10:1 import-extcommunity
 vpn-target 10:1 export-extcommunity
 quit
interface M-GigabitEthernet0/0/0
 description mgmt_oobm
 ip binding vpn-instance MGMT
 ip address <MGMT-IP-TX-K> <MASK>
 quit
ip route-static vpn-instance MGMT 0.0.0.0 0 <MGMT-GW> description mgmt-DEFAULT-GATEWAY
ntp-service enable
ntp-service source M-GigabitEthernet0/0/0
ntp-service unicast-server 10.0.31.21 vpn-instance MGMT
ntp-service unicast-server 10.0.31.23 vpn-instance MGMT
info-center loghost vpn-instance MGMT 10.0.61.10
ssh server enable
lldp global enable
# SNMPv3, local users and AAA: copy the LVR standard from an existing Leaf
#
# --- no spanning tree, no production VLANs ---
undo stp global enable
#
# --- S-VLANs: one per attachment point, plus the stretched FW zone (option A; no member port with option B) ---
vlan 3101 to 3110
vlan 3201 to 3210
vlan 3101
 name TRX-AP-K-L11
 quit
vlan 3102
 name TRX-AP-K-L12
 quit
vlan 3103
 name TRX-AP-K-L21
 quit
vlan 3104
 name TRX-AP-K-L31
 quit
vlan 3105
 name TRX-AP-K-L41
 quit
vlan 3110
 name TRX-AP-K-BL
 quit
vlan 3300
 name TRX-AP-FW-ZONE-stretched
 quit
#
# --- attachment-point ports: QinQ access, PVID = S-VLAN of the AP ---
interface Ten-GigabitEthernet1/1/1
 description TRX AP K-L11 --> sw-rzk-L11 XGE1/3/1
 port link-type access
 port access vlan 3101
 qinq enable
# bpdu-drop any
 quit
interface Ten-GigabitEthernet1/1/2
 description TRX AP K-L12 --> sw-rzk-L12 <PORT>
 port link-type access
 port access vlan 3102
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/3
 description TRX AP K-L21 --> sw-rzk-L21 <PORT>
 port link-type access
 port access vlan 3103
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/4
 description TRX AP K-L31 --> sw-rzk-L31 <PORT>
 port link-type access
 port access vlan 3104
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/5
 description TRX AP K-L41 --> sw-rzk-L41 <PORT>
 port link-type access
 port access vlan 3105
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/6
 description TRX AP K-BL --> sw-rzk-BL XGE1/3/6
 port link-type access
 port access vlan 3110
 qinq enable
 quit
# firewall zone option A only: direct link to the cluster member. With option B (default) the port stays shut down.
interface Ten-GigabitEthernet1/1/7
 description TRX AP FW-ZONE (option A) --> CPF-ZV01 <FW-PORT>
 port link-type access
 port access vlan 3300
 qinq enable
 shutdown
 quit
#
# --- monitoring trunks to trex-mon p0-p3: frames stay double-tagged ---
interface Ten-GigabitEthernet1/1/21
 description TRX p0 trex-mon (double-tagged, all APs)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3101 to 3110 3201 to 3210 3300
 quit
interface Ten-GigabitEthernet1/1/22
 description TRX p1 trex-mon (double-tagged, all APs)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3101 to 3110 3201 to 3210 3300
 quit
interface Ten-GigabitEthernet1/1/23
 description TRX p2 trex-mon reserve
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3101 to 3110 3201 to 3210 3300
 shutdown
 quit
interface Ten-GigabitEthernet1/1/24
 description TRX p3 trex-mon reserve
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3101 to 3110 3201 to 3210 3300
 shutdown
 quit
#
# --- lab ports to trex-lab p4-p7: QinQ access, PVID = the pinned AP ---
interface Ten-GigabitEthernet1/1/9
 description TRX p4 trex-lab pinned [K-L11]
 port link-type access
 port access vlan 3101
 qinq enable
# mac-address static 0254-52f0-0004 vlan 3101
# unicast-suppression pps <MAX-PPS>
 quit
interface Ten-GigabitEthernet1/1/10
 description TRX p5 trex-lab pinned [K-L41]
 port link-type access
 port access vlan 3105
 qinq enable
# mac-address static 0254-52f0-0005 vlan 3105
 quit
interface Ten-GigabitEthernet1/1/11
 description TRX p6 trex-lab pinned [FW-ZONE via K-BL, VLAN 4003]
 port link-type access
 port access vlan 3110
 qinq enable
# mac-address static 0254-52f0-0006 vlan 3110
 quit
# option A instead: port access vlan 3300 (AP FW)
interface Ten-GigabitEthernet1/1/12
 description TRX p7 trex-lab pinned [C-L21]
 port link-type access
 port access vlan 3204
 qinq enable
# mac-address static 0254-52f0-0007 vlan 3204
 quit
#
# --- TX-to-TX link (dark fibre / DWDM): RZC S-VLANs, plus the stretched zone S-VLAN ---
interface HundredGigE1/2/1
 description TRX inter-DC --> sw-rzc-TX1 HGE1/2/1 (dark fibre)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3201 to 3210 3300
 quit
#
# --- unused ports administratively down ---
interface range Ten-GigabitEthernet1/1/8 Ten-GigabitEthernet1/1/13 to Ten-GigabitEthernet1/1/20
 shutdown
 quit
return
save force

Line	What it does	Why
`undo stp global enable`	disables spanning tree globally (L2CG p. 306, L2CR p. 321)	each S-VLAN holds one AP plus TRex ports, so no loop can form and the TX switch must not take part in any spanning tree (the examined fabric switches have STP globally disabled). With "initial settings" STP starts disabled, with "factory defaults" enabled - so configure it explicitly
`vlan 3101 to 3110`, `vlan 3201 to 3210`, `vlan 3300`	creates the S-VLANs in bulk (L2CR p. 200)	3201-3210 are needed here as well: the RZC APs reach the server through this switch. 3300 is the stretched zone S-VLAN of option A - with option B it exists without an active member port
`name ...`	VLAN name, at most 32 characters (L2CR p. 197)	readable `display vlan` output
`port link-type access`, `port access vlan 3101`	access port with PVID 3101 (L2CR pp. 202, 204)	the PVID is the S-VLAN that identifies the AP
`qinq enable`	tags every incoming frame, tagged or untagged, with the PVID and removes that tag on egress (L2CG pp. 220-222, L2CR p. 278)	the Leaf delivers single-tagged production VLANs; towards TRex they become double-tagged; default TPID 0x8100 for both tags (L2CG p. 223)
`XGE1/1/7`: access, PVID 3300, `qinq enable`, `shutdown`	the option-A firewall port of CPF-ZV01, configured but administratively down (L2CR pp. 96-97)	with option B nothing is cabled here and the port stays down; if option A is chosen, only the cabling and `undo shutdown` are missing (block below)
trunk with `undo port trunk permit vlan 1` and `port trunk permit vlan 3101 to 3110 3201 to 3210 3300`	802.1Q trunk towards `trex-mon` (L2CR pp. 204-206)	frames keep both tags, so every AP - and through K-BL and C-BL also the firewall zone (C-VLAN 4003) - is reachable from every monitoring port without re-cabling. 3300 on the trunk is harmless with option B (no member ports) and required with option A
lab ports: access + `qinq enable`, PVID = pinned AP	TRex sends single-tagged frames, the switch adds and removes the S-tag	X710 hardware flow statistics support at most one VLAN tag; re-pinning is one `port access vlan` command - the port MAC and the TRex configuration do not change, only the lab IP follows the new AP
p5 `port access vlan 3105`, description `[K-L41]`	pins p5 to AP K-L41	default pin of the plan, so that p5 no longer duplicates p6's S-VLAN; the L3-BL line-rate test uses p4 (K-L11, VLAN 1314) and p6 (K-BL, which also carries VLAN 1315)
p6 `port access vlan 3110`, description `[FW-ZONE via K-BL, VLAN 4003]`	pins p6 to `FWZ`, the firewall zone (VLAN 4003) as reached through AP K-BL	with option B the zone has no AP of its own: TRex sends VLAN 4003 and the switch pushes S-VLAN 3110. p6 is the only lab port in 3110 (`display vlan 3110` shows `XGE1/1/6` and `XGE1/1/11` untagged). Option A moves p6 to PVID 3300 (below)
`HundredGigE1/2/1` trunk `3201 to 3210 3300`	carries the RZC S-VLANs over the dedicated fibre	the server is in RZK; measurements never depend on the fabric for the way back. 3300 is harmless with option B and required with option A, where both firewall members must share one Layer 2 domain
`interface range ... shutdown`	shuts the unused ports (L2CR pp. 34-35)	no accidental connections into an S-VLAN; `XGE1/1/8` and `XGE1/1/13-20` are unused, `XGE1/1/7` is shut down in its own interface block
`info-center loghost vpn-instance MGMT 10.0.61.10`	syslog host reached through VPN MGMT (Network Management and Monitoring Command Reference, PDF p. 440)	the `vpn-instance` keyword stands before the address
`ntp-service unicast-server ... vpn-instance MGMT`	NTP through VPN MGMT (same reference, PDF p. 140)	same servers and style as sw-rzk-L11
`save force`	saves without a confirmation prompt (FCR p. 189)
`# bpdu-drop any` (optional, AP ports)	drops BPDUs received on the port (L2CR pp. 292-293)	only if bench tests show that Leaf BPDUs reach the TRex ports and that this is a problem
`# mac-address static <lab-port-MAC> vlan <S-VLAN>` (optional, lab ports)	static MAC entry in interface view; the VLAN must be one the port belongs to (L2CR pp. 16-17)	one entry per lab port for its fixed port MAC in the S-VLAN of its current pin (`0254-52f0-0004` = p4 at K-L11 in 3101, `0254-52f0-0005` = p5 at K-L41 in 3105, `0254-52f0-0006` = p6, the zone lab client 10.9.90.156, in 3110, `0254-52f0-0007` = p7 at C-L21 in 3204). The MAC is always known on its lab port, so return traffic of a lab test is never flooded into the monitoring trunks. The entry does not depend on the VLAN or the IP, because TRex answers ARP and runs ASTF with the port MAC; it moves with the pin
`# unicast-suppression pps <MAX-PPS>` (optional)	unknown-unicast storm suppression (L2CR p. 98)	caps what a lab test can flood; whether it acts on received or sent traffic must be confirmed on the bench

Lab tests and unknown-unicast flooding

Each S-VLAN also contains the monitoring trunks. A lab frame whose destination MAC the TX switch has not learned is flooded to the AP and to trex-mon (software RX), and the Leaf floods it in the production VSI. The static entries above protect the return direction. For the forward direction, learn the remote lab MAC before the test (bidirectional traffic or a gratuitous ARP), or add a static entry for the remote lab client on the port that leads to its AP (the AP port, or HGE1/2/1 for an RZC AP) confirm on device. Re-pinning a lab port means moving its static entries as well. See Chapter 24.

Firewall-zone option A adds (sw-rzk-TX1) - only if option A is chosen

With option A the zone is a new firewall port per cluster member on the TX switches. The default configuration above already contains everything except the cabling: S-VLAN 3300, the port of CPF-ZV01 on XGE1/1/7 (access, PVID 3300, QinQ, shutdown) and 3300 on the monitoring trunks and on the TX-to-TX link. Option A therefore adds only the firewall cabling, undo shutdown on the firewall port and the re-pin of p6 to the zone AP FW.

verified syntax (L2CR pp. 16, 96-97, 202) · decision only with firewall-zone option A

sw-rzk-TX1 - firewall-zone option A adds

system-view
# cable CPF-ZV01 <FW-PORT> to XGE1/1/7, then bring up the pre-configured port
interface Ten-GigabitEthernet1/1/7
 undo shutdown
 quit
# lab port p6 pinned to the zone AP FW
interface Ten-GigabitEthernet1/1/11
# undo mac-address static 0254-52f0-0006 vlan 3110
 description TRX p6 trex-lab pinned [FW-ZONE]
 port access vlan 3300
# mac-address static 0254-52f0-0006 vlan 3300
 quit
return
save force

S-VLAN 3300 is the one S-VLAN with two AP ports - the firewall ports of CPF-ZV01 (here) and CPF-ZV02 (on sw-rzc-TX1), joined over the TX-to-TX link because both members of the cluster must see the same zone segment. That is safe only because the zone VLAN 1420 exists on the firewall interfaces alone, never in the fabric, and a routed firewall does not bridge between its interfaces (Chapter 12). undo shutdown (L2CR pp. 96-97) brings up the port that the default configuration keeps down. The lab zone client becomes 10.9.90.164 in S-VLAN 3300 - still with p6's fixed port MAC 0254-52f0-0006; a static entry of the default configuration is removed first with the undo form (L2CR p. 16), because a static entry needs a VLAN that the port belongs to. The ClusterXL dependency that comes with these ports is described in Chapter 15 and Chapter 24.

Variant for 5950 32QSFP28 chassis (JH321A): split a 100-GE port into four 10-GE breakout interfaces with a dedicated breakout cable; no reboot is needed (L2CG pp. 37-38). verified syntax · confirm on device cable type.

Breakout variant (JH321A)

interface HundredGigE1/0/1
 using tengige
 quit
interface Ten-GigabitEthernet1/0/1:1
 description TRX AP K-L11 --> sw-rzk-L11 XGE1/3/1
 port link-type access
 port access vlan 3101
 qinq enable
 quit

No MTU change is needed for the four extra bytes of the S-tag: 5950 Ethernet interfaces accept frames up to 9416 bytes by default (jumboframe enable, L2CG p. 41).

B.4 TX switch sw-rzc-TX1¶

Same base configuration (system, management, NTP, syslog, SSH, LLDP, undo stp global enable) with sysname sw-rzc-TX1 and its own management address. There are no TRex ports in RZC: the AP ports reach the server through the TX-to-TX link. As on sw-rzk-TX1, the configuration below is the default with firewall-zone option B: the option-A firewall port XGE1/1/8 (CPF-ZV02) is configured but shut down, and S-VLAN 3300 is defined and permitted on the TX-to-TX link without an active member port decision.

verified syntax · confirm on device interface numbers and remote ports · source: plan v0.92 sections 6.2 and 7.2, completed for all ports

sw-rzc-TX1 - port configuration (option B, the default; base configuration as in B.3)

system-view
 sysname sw-rzc-TX1
# ... base configuration as on sw-rzk-TX1, management address <MGMT-IP-TX-C> ...
undo stp global enable
vlan 3201 to 3210
vlan 3201
 name TRX-AP-C-L11
 quit
vlan 3202
 name TRX-AP-C-L12
 quit
vlan 3203
 name TRX-AP-C-L13
 quit
vlan 3204
 name TRX-AP-C-L21
 quit
vlan 3205
 name TRX-AP-C-L22
 quit
vlan 3206
 name TRX-AP-C-L23
 quit
vlan 3210
 name TRX-AP-C-BL
 quit
vlan 3300
 name TRX-AP-FW-ZONE-stretched
 quit
#
interface Ten-GigabitEthernet1/1/1
 description TRX AP C-L11 --> sw-rzc-L11 <PORT>
 port link-type access
 port access vlan 3201
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/2
 description TRX AP C-L12 --> sw-rzc-L12 <PORT>
 port link-type access
 port access vlan 3202
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/3
 description TRX AP C-L13 --> sw-rzc-L13 <PORT>
 port link-type access
 port access vlan 3203
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/4
 description TRX AP C-L21 --> SW-RZC-L21 XGE1/4/4
 port link-type access
 port access vlan 3204
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/5
 description TRX AP C-L22 --> sw-rzc-L22 <PORT>
 port link-type access
 port access vlan 3205
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/6
 description TRX AP C-L23 --> sw-rzc-L23 <PORT>
 port link-type access
 port access vlan 3206
 qinq enable
 quit
interface Ten-GigabitEthernet1/1/7
 description TRX AP C-BL --> sw-rzc-BL XGE1/3/14
 port link-type access
 port access vlan 3210
 qinq enable
 quit
# firewall zone option A only: direct link to the cluster member. With option B (default) the port stays shut down.
interface Ten-GigabitEthernet1/1/8
 description TRX AP FW-ZONE (option A) --> CPF-ZV02 <FW-PORT>
 port link-type access
 port access vlan 3300
 qinq enable
 shutdown
 quit
#
interface HundredGigE1/2/1
 description TRX inter-DC --> sw-rzk-TX1 HGE1/2/1 (dark fibre)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 3201 to 3210 3300
 quit
#
interface range Ten-GigabitEthernet1/1/9 to Ten-GigabitEthernet1/1/24
 shutdown
 quit
return
save force

The TX-to-TX link carries the RZC S-VLANs 3201-3210 and the stretched zone S-VLAN 3300 (harmless with option B, where 3300 has no member port; required with option A, where both firewall members must share one Layer 2 domain); it does not need the RZK S-VLANs. Its speed and optics depend on the fibre distance (open decision, Appendix D). If the link is 10G, use a Ten-GigabitEthernet port with the same trunk configuration.

Firewall-zone option A adds (sw-rzc-TX1) - only if option A is chosen

On sw-rzc-TX1 the default configuration already holds S-VLAN 3300, the firewall port of CPF-ZV02 on XGE1/1/8 (shut down) and 3300 on the TX-to-TX link. Option A adds only the cabling and undo shutdown; together with the option-A lines of B.3 the zone segment is then stretched to the second member and to the monitoring trunks in RZK.

verified syntax (L2CR pp. 96-97) · decision only with firewall-zone option A

sw-rzc-TX1 - firewall-zone option A adds

system-view
# cable CPF-ZV02 <FW-PORT> to XGE1/1/8, then bring up the pre-configured port
interface Ten-GigabitEthernet1/1/8
 undo shutdown
 quit
return
save force

B.5 Attachment-point ports on Leafs and Border Leafs¶

The AP port is configured exactly like an existing server trunk: 802.1Q trunk, one Ethernet service instance per VLAN under test, cross-connected to the existing VSI. The pattern is the one of sw-rzk-L11 Ten-GigabitEthernet1/3/3 ("-> ESH-ZV12 Trunk") in the running configuration of 17 February 2025. Explained in Chapter 13.

verified pattern (LVR running configurations; VXCR pp. 24, 42, 54-55) · confirm on device that the port is still unused and the VSIs exist · source: plan v0.92 sections 7.3 and 7.4

Template - AP port on a Leaf or Border Leaf

interface <LEAF-PORT>
 port link-mode bridge
 description TRX AP <AP-CODE> --> <TX-SWITCH> <TX-PORT> (TRex attachment point)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 1314 to 1315 <PROD-VLAN-A> <PROD-VLAN-B>
# stp edged-port
 service-instance 1314
  encapsulation s-vid 1314
  xconnect vsi VPN-1314
  quit
 service-instance 1315
  encapsulation s-vid 1315
  xconnect vsi VPN-1315
  quit
 service-instance <PROD-VLAN-A>
  encapsulation s-vid <PROD-VLAN-A>
  xconnect vsi VPN-<PROD-VLAN-A>
  quit
 service-instance <PROD-VLAN-B>
  encapsulation s-vid <PROD-VLAN-B>
  xconnect vsi VPN-<PROD-VLAN-B>
  quit
 quit

Line	What it does	Why
`port link-type trunk`, `undo port trunk permit vlan 1`, `port trunk permit vlan ...`	trunk carrying only the VLANs under test	same as the existing server ports; VLAN 1 is never used
`service-instance N` / `encapsulation s-vid N`	Ethernet service instance matching frames tagged with VLAN N (VXCR pp. 42, 24)	the Leafs map VLANs to VSIs per port, not globally
`xconnect vsi VPN-N`	maps the instance to the VSI; default access mode VLAN: the tag is removed towards the VXLAN and added towards the port (VXCR pp. 54-55)	identical behaviour to every other server port
`# stp edged-port` (optional)	edge port (L2CG p. 294, L2CR p. 318)	has no effect where spanning tree is globally disabled (all examined switches: `display stp` "Protocol status: Disabled") and is not used on the existing server ports - add it only on a Leaf that runs PVST for a monitored VLAN

sw-rzk-L11 - AP K-L11 (Ten-GigabitEthernet1/3/1, free in February 2025)

system-view
interface Ten-GigabitEthernet1/3/1
 port link-mode bridge
 description TRX AP K-L11 --> sw-rzk-TX1 XGE1/1/1 (TRex attachment point)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 1314 to 1315
 service-instance 1314
  encapsulation s-vid 1314
  xconnect vsi VPN-1314
  quit
 service-instance 1315
  encapsulation s-vid 1315
  xconnect vsi VPN-1315
  quit
 quit
save force

SW-RZC-L21 - AP C-L21 (Ten-GigabitEthernet1/4/4, free in February 2025)

system-view
interface Ten-GigabitEthernet1/4/4
 port link-mode bridge
 description TRX AP C-L21 --> sw-rzc-TX1 XGE1/1/4 (TRex attachment point)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 1314 to 1315
 service-instance 1314
  encapsulation s-vid 1314
  xconnect vsi VPN-1314
  quit
 service-instance 1315
  encapsulation s-vid 1315
  xconnect vsi VPN-1315
  quit
 quit
save force

sw-rzk-BL - AP K-BL (Ten-GigabitEthernet1/3/6, free in February 2025)

system-view
interface Ten-GigabitEthernet1/3/6
 port link-mode bridge
 description TRX AP K-BL --> sw-rzk-TX1 XGE1/1/6 (TRex attachment point)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 1314 to 1315 4003
 service-instance 1314
  encapsulation s-vid 1314
  xconnect vsi VPN-1314
  quit
 service-instance 1315
  encapsulation s-vid 1315
  xconnect vsi VPN-1315
  quit
# firewall zone (option B): the existing transfer VSI 4003 becomes the TRex zone
 service-instance 4003
  encapsulation s-vid 4003
  xconnect vsi VPN-4003
  quit
 quit
save force

sw-rzc-BL - AP C-BL (Ten-GigabitEthernet1/3/14, free in February 2025)

system-view
interface Ten-GigabitEthernet1/3/14
 port link-mode bridge
 description TRX AP C-BL --> sw-rzc-TX1 XGE1/1/7 (TRex attachment point)
 port link-type trunk
 undo port trunk permit vlan 1
 port trunk permit vlan 1314 to 1315 4003
 service-instance 1314
  encapsulation s-vid 1314
  xconnect vsi VPN-1314
  quit
 service-instance 1315
  encapsulation s-vid 1315
  xconnect vsi VPN-1315
  quit
# firewall zone (option B): the existing transfer VSI 4003 becomes the TRex zone
 service-instance 4003
  encapsulation s-vid 4003
  xconnect vsi VPN-4003
  quit
 quit
save force

SW-RZC-L21 runs R6205P03; its own running configuration uses the same service-instance / xconnect vsi syntax. Its port XGE1/4/4 was down and in default state in February 2025, but has an HPE transceiver inserted and may be reserved confirm on device.

Firewall zone: VLAN 4003 on the Border Leaf APs (option B, the default)

The firewall zone is the existing, free transfer VSI 4003 "Transfer ENTFW *frei*": the firewall team adds a VLAN 4003 interface (VIP 10.9.90.1) on the bond that already connects both cluster members to the Border Leafs, and TRex reaches the zone through the Border Leaf attachment points K-BL and C-BL like any other VLAN. VSI VPN-4003 (vxlan 4003, no gateway interface) exists on both Border Leafs - not on the Leafs examined - and is already a service instance on Bridge-Aggregation10 towards each firewall member (running configurations of February 2025). The BL AP ports therefore carry 4003 in port trunk permit vlan and a third service instance; the zone clients sit behind K-BL (10.9.90.106) and C-BL (10.9.90.113), the lab zone client behind p6 (10.9.90.156).

The plan recommends option B and uses it as the default; option A - a new firewall port per member on the TX switches, the customer's original wish - remains the alternative decision. With option A the 4003 entries are omitted. See B.6.

Pre-checks before the change (all display commands, see A.6): display current-configuration interface Ten-GigabitEthernet1/3/1 (port really in default state), display interface Ten-GigabitEthernet1/3/1 (down, no traffic), display l2vpn vsi name VPN-1314 verbose (VSI exists - not every VSI exists on every Leaf, so check each VLAN on each Leaf; on the Border Leafs also VPN-4003).

Commissioning checks after the change confirm on device (Chapter 21, scenario S1):

Gateway MAC on the RZC Leafs. display l2vpn mac-address vsi VPN-1314 on every RZC Leaf with an AP must list ec9b-8b67-37b8, the gateway MAC of Vsi-interface914/915 on sw-rzk-BL (VXCR p. 13). SW-RZC-L21 had no EVPN entry for it in February 2025; without it the routed streams of that AP are flooded as unknown unicast and arrive as duplicates at the other RZC APs.
ARP through the tunnels. The EMU clients behind the Leaf APs must resolve 10.3.14.1 / 10.3.15.1 (centralised gateway, EVPN Configuration Guide R630x, PDF p. 28). If they do not, the fallbacks are arp suppression enable in the VSI view of the Leafs (VXCR pp. 5-6, a production change) or the L3 classes from the Border Leaf APs decision.

Rollback of an AP port

system-view
interface Ten-GigabitEthernet1/3/1
 undo service-instance 1314
 undo service-instance 1315
 default
 quit
save force

default restores the interface defaults and asks for confirmation (L2CR p. 41). On a Border Leaf AP port remove service-instance 4003 the same way (undo service-instance 4003) before default. The VSIs stay untouched. Take a display current-configuration snapshot before and after every change.

B.6 Firewall test zone requirements¶

The firewall zone gives the L3-ENTFW class its destination: VLAN 1314 (VPN-CORE) -> sw-rzk-BL -> Vsi40 -> ENT-FW -> TRex zone. The firewall zone is the existing, free transfer VSI 4003: the firewall team adds a VLAN 4003 interface (VIP 10.9.90.1) on the bond that already connects both cluster members to the Border Leafs, and TRex reaches the zone through the Border Leaf attachment points K-BL and C-BL like any other VLAN. The plan recommends this option B and uses it as the default; option A - a new firewall port per member on the TX switches, the customer's original wish - remains the alternative decision. The firewall configuration itself is done by the LVR firewall team; this section lists what the platform needs. Explained in Chapter 15.

decision (option B default, option A alternative) · source: Chapter 15 and plan v0.92 section 7.5

Item	Option B - existing transfer VSI 4003 (default, recommended by the plan)	Option A - new physical firewall interface through the TX switches (alternative)
Firewall interface	none new: a VLAN 4003 sub-interface on the existing inside bond of both members (BAGG10 "Channel to ENT-FW ZV01/ZV02 inside" on the Border Leafs, LACP, trunk VLANs 4000-4003)	one new 10G port per cluster member, cabled to sw-rzk-TX1 `XGE1/1/7` (ZV01) and sw-rzc-TX1 `XGE1/1/8` (ZV02); two Check Point certified SFP+ SR optics on the firewall side and one more SR optic per TX switch
VLAN	VLAN 4003: VSI `VPN-4003` "Transfer ENTFW frei" (vxlan 4003, no gateway interface, no MACs learned in February 2025)	VLAN 1420 sub-interface on the new port of both members (e.g. eth1-05.1420)
TX switches	no change and nothing to cable: S-VLAN 3300 is defined and permitted on the trunks, `XGE1/1/7` (rzk) and `XGE1/1/8` (rzc) carry the option-A port configuration but stay shut down (B.3, B.4)	firewall cabling and `undo shutdown` on `XGE1/1/7` and `XGE1/1/8` (QinQ access ports with PVID 3300, already configured); lab port p6 re-pinned to PVID 3300 (the option-A blocks in B.3 and B.4)
Fabric	VLAN 4003 with a service instance 4003 on the Border Leaf AP ports of K-BL and C-BL (B.5)	no change
Addresses (proposal, IPAM approval needed)	10.9.90.0/24, cluster VIP 10.9.90.1, members .2/.3, TRex pool 10.9.90.101-116, lab pool .151-.166; zone clients at K-BL {p0, S 3110, C 4003} 10.9.90.106 and at C-BL {p1, S 3210, C 4003} 10.9.90.113; lab zone client on p6 (pinned to `FWZ`) 10.9.90.156 with p6's fixed port MAC `02:54:52:f0:00:06`	same subnet and pools; zone client at AP FW (index 14) {p0, S 3300, C 1420} 10.9.90.114; lab zone client 10.9.90.164, same port MAC
Monitoring pairs (`l3-entfw`)	sources at K-L11 and K-BL to the zone client at K-BL, sources at C-L21 and C-BL to the zone client at C-BL (the zone client sits at the BL AP of the same DC)	all four sources to the zone client at AP FW
Failure behaviour	no new dependency of the cluster on the TX switches or the TX-to-TX fibre: a TX-switch failure cannot affect production	ClusterXL monitors cluster interfaces: a TX switch reboot, fibre cut, optic or TRex port failure in this non-redundant path can put a member into a problem state (failover or loss of redundancy) unless the firewall team makes the interface non-critical for their Check Point version or formally accepts the risk
Drawbacks	not the "physical firewall link from the TX switch" originally requested; the zone leg also crosses the fabric (a few tens of microseconds, the same for all zone streams); K-BL and C-BL also carry VLAN 4003	the TX switches become relevant for the firewall cluster; new firewall ports and optics

Both options use the same subnet, the same address formula, the same firewall objects and rules and the same trexctl group (l3-entfw); only the inventory entry of the zone and the TX-switch or Border Leaf configuration differ. The lab zone client keeps p6's fixed port MAC in either option; only its IP differs (.156 / .164).

Common to both options:

Item	Requirement
Topology / anti-spoofing	internal, network defined by the interface address and mask; anti-spoofing on (prevent and log)
Routing	the firewall advertises 10.9.90.0/24 into OSPF 1 of VPN-CORE (it is already a neighbour on Vsi40); alternative: static route on both Border Leafs (below). VPN-CORE already has a default route 0.0.0.0/0 via 10.9.0.22, so the zone is reachable even before that
Host objects	TRX-mon-1314 (10.3.14.101-116), TRX-lab-1314 (10.3.14.151-166), TRX-mon-1315 (10.3.15.101-116), TRX-mon-<VLAN> per production pool, TRX-fwzone (10.9.90.101-116), TRX-fwzone-lab (10.9.90.151-166); groups TRX-inside and TRX-zone
Services	TRX-udp = UDP 32000-32099; TRX-tcp = TCP 80, 443, 8080 (ASTF); ICMP echo
Rules	1) TRX-inside -> TRX-zone and 2) TRX-zone -> TRX-inside: TRX-udp, ICMP, accept, log; 3) ASTF: TRX-astf-clients (10.9.92.0/24 or the lab client 10.3.14.151) -> TRX-astf-servers (10.9.93.0/24 or the zone lab client 10.9.90.156; option A 10.9.90.164): TRX-tcp, accept. Above any cleanup or drop rule
ASTF variant 2 (routed ranges)	firewall route 10.9.93.0/24 via the zone lab client 10.9.90.156 (option A: 10.9.90.164); the client range 10.9.92.0/24 is routed on sw-rzk-BL (below)
NAT	no NAT for TRex objects
Threat Prevention	inspect TRex traffic or not - open decision
Timeouts	Check Point UDP virtual session timeout 40 s by default; the continuous streams keep their sessions alive
Cluster MAC and failover	in February 2025 both clusters answered ARP with member MACs (no virtual MAC seen). Streams with a firewall destination MAC - all L3-DCFW streams and the zone-side `l3-entfw` and `dscp-entfw` streams - keep the old MAC after a failover and lose packets until they are rebuilt. The planned `trexctl` gateway-MAC tracker polls `dgw.rmac` every 10 s for every VLAN with `gateway_mac_tracking: true` and rebuilds the affected streams (event `gateway_mac_changed`). How fast EMU learns the new MAC depends on how ClusterXL announces it: a gratuitous ARP request is learned at once, a broadcast ARP reply is ignored and the next periodic query (every 60 s by default) is needed - the plan's target of 15 s therefore needs a shorter ARP timer for the firewall VLANs or a forced query confirm on device. A ClusterXL virtual MAC would avoid the rebuild - decision with the firewall team; test during a planned failover
Packet properties	latency streams carry UDP checksum 0; TRex sets bit 0 of the TOS byte of all statistics streams (EF 0xB8 leaves as 0xB9) - confirm on device that the firewalls pass both unchanged
DC firewall (class L3-DCFW)	rules TRX-mon-<VLAN-A> <-> TRX-mon-<VLAN-B>, TRX-udp and ICMP, no NAT; no interface change

Static alternative on sw-rzk-BL and sw-rzc-BL (only if the firewall does not advertise the zone)

ip route-static vpn-instance VPN-CORE 10.9.90.0 24 10.9.0.22 description TRex-zone

Optional, ASTF variant 2 only (sw-rzk-BL): routed client range behind the lab client 10.3.14.151

ip route-static vpn-instance VPN-CORE 10.9.92.0 24 10.3.14.151 description TRex-ASTF-clients

verified syntax (identical style to the existing VPN-CORE static routes on both Border Leafs; Layer 3 - IP Routing Command Reference, PDF pp. 84-85) · decision prefixes. The existing VPN-CORE static routes carry preference 70 on sw-rzc-BL and no preference on sw-rzk-BL - follow the style of each device. OSPF 1 of VPN-CORE has import-route static, so both routes are redistributed. Remove the ASTF route after the test campaign with undo ip route-static vpn-instance VPN-CORE 10.9.92.0 24 10.3.14.151.

B.7 trexctl inventory (planned format)¶

planned format · decision (firewall zone: option B shown, option A as comments; production VLANs) · source: plan v0.92 section 9.3; budget as in the monitoring design of Chapter 16. Explained in Chapter 18.

/etc/trexctl/inventory.yaml - planned format, trexctl does not exist yet

version: 1
site:
  name: LVR RZK/RZC
  mac_oui: "02:54:52"            # locally administered prefix of TRex clients
trex:
  mon: { server: 127.0.0.1, rpc_port: 4501, pub_port: 4500, emu_port: 4510,
         mode: stl-software-emu }
  lab: { server: 127.0.0.1, rpc_port: 4601, pub_port: 4600, mode: stl-hw }
  # trex-lab-astf uses the same ports and its own /etc/trex/trex_lab_astf.yaml:
  # p4 = TRex port 0 (client side), p6 = TRex port 1 (server side)
ports:                           # keys = TRex port numbers as each instance counts them
  mon:                           # trex-mon ports 0-3 = p0-p3
    0: { type: multiplexed, tx_switch: sw-rzk-TX1, tx_port: XGE1/1/21,
         default_for: [rzk] }
    1: { type: multiplexed, tx_switch: sw-rzk-TX1, tx_port: XGE1/1/22,
         default_for: [rzc] }
  lab:                           # trex-lab ports 0-3; label = server-wide name p4-p7
    0: { label: p4, mac: "02:54:52:f0:00:04", type: pinned,
         tx_switch: sw-rzk-TX1, tx_port: XGE1/1/9,  ap: K-L11, astf_role: client }
    1: { label: p5, mac: "02:54:52:f0:00:05", type: pinned,
         tx_switch: sw-rzk-TX1, tx_port: XGE1/1/10, ap: K-L41 }
    2: { label: p6, mac: "02:54:52:f0:00:06", type: pinned,
         tx_switch: sw-rzk-TX1, tx_port: XGE1/1/11, ap: FWZ, astf_role: server }   # option A: ap: FW
    3: { label: p7, mac: "02:54:52:f0:00:07", type: pinned,
         tx_switch: sw-rzk-TX1, tx_port: XGE1/1/12, ap: C-L21 }
# the lab-port MACs must match src_mac in /etc/trex/trex_lab.yaml and trex_lab_astf.yaml (B.1)
vlans_under_test: &utv [1314, 1315, <PROD-VLAN-A>, <PROD-VLAN-B>]   # carried by every Leaf/BL AP
aps:
  K-L11: { index: 1, dc: rzk, device: sw-rzk-L11,
           device_port: Ten-GigabitEthernet1/3/1, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/1, svlan: 3101, vlans: *utv }
  K-L12: { index: 2, dc: rzk, device: sw-rzk-L12,
           device_port: TBD, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/2, svlan: 3102, vlans: *utv }
  K-L21: { index: 3, dc: rzk, device: sw-rzk-L21,
           device_port: TBD, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/3, svlan: 3103, vlans: *utv }
  K-L31: { index: 4, dc: rzk, device: sw-rzk-L31,
           device_port: TBD, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/4, svlan: 3104, vlans: *utv }
  K-L41: { index: 5, dc: rzk, device: sw-rzk-L41,
           device_port: TBD, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/5, svlan: 3105, vlans: *utv }
  K-BL:  { index: 6, dc: rzk, device: sw-rzk-BL,
           device_port: Ten-GigabitEthernet1/3/6, tx_switch: sw-rzk-TX1,
           tx_port: XGE1/1/6, svlan: 3110,
           vlans: [1314, 1315, <PROD-VLAN-A>, <PROD-VLAN-B>, 4003] }
  C-L11: { index: 7, dc: rzc, device: sw-rzc-L11,
           device_port: TBD, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/1, svlan: 3201, vlans: *utv }
  C-L12: { index: 8, dc: rzc, device: sw-rzc-L12,
           device_port: TBD, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/2, svlan: 3202, vlans: *utv }
  C-L13: { index: 9, dc: rzc, device: sw-rzc-L13,
           device_port: TBD, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/3, svlan: 3203, vlans: *utv }
  C-L21: { index: 10, dc: rzc, device: SW-RZC-L21,
           device_port: Ten-GigabitEthernet1/4/4, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/4, svlan: 3204, vlans: *utv }
  C-L22: { index: 11, dc: rzc, device: sw-rzc-L22,
           device_port: TBD, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/5, svlan: 3205, vlans: *utv }
  C-L23: { index: 12, dc: rzc, device: sw-rzc-L23,
           device_port: TBD, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/6, svlan: 3206, vlans: *utv }
  C-BL:  { index: 13, dc: rzc, device: sw-rzc-BL,
           device_port: Ten-GigabitEthernet1/3/14, tx_switch: sw-rzc-TX1,
           tx_port: XGE1/1/7, svlan: 3210,
           vlans: [1314, 1315, <PROD-VLAN-A>, <PROD-VLAN-B>, 4003] }
  # firewall zone, option B (default): VSI 4003 reached through the BL APs - no AP of
  # its own; K-BL and C-BL carry VLAN 4003. FWZ is the alias used by the lab pins.
  FWZ:   { alias_of: K-BL, vlan: 4003 }
  # firewall zone, option A (alternative): direct links on the pre-configured, shut-down
  # TX ports XGE1/1/7 (rzk) and XGE1/1/8 (rzc), own AP index 14
  # FW:  { index: 14, dc: both, device: CPF-ZV01/ZV02, device_port: TBD,
  #        tx_switch: both, tx_port: XGE1/1/7+XGE1/1/8, svlan: 3300,
  #        vlans: [1420] }
vlans:                           # client IP = pool_start + (index - 1); lab IP = lab_pool_start + (index - 1)
  1314: { vsi: VPN-1314, prefix: 10.3.14.0/24, gateway: 10.3.14.1,
          gateway_owner: sw-rzk-BL/VPN-CORE, pool_start: 10.3.14.101,
          lab_pool_start: 10.3.14.151, vrf: VPN-CORE }
  1315: { vsi: VPN-1315, prefix: 10.3.15.0/24, gateway: 10.3.15.1,
          gateway_owner: sw-rzk-BL/VPN-CORE, pool_start: 10.3.15.101,
          lab_pool_start: 10.3.15.151, vrf: VPN-CORE }
  4003: { vsi: VPN-4003, prefix: 10.9.90.0/24, gateway: 10.9.90.1,
          gateway_owner: ENT-FW, pool_start: 10.9.90.101,
          lab_pool_start: 10.9.90.151, vrf: FW-ZONE,
          gateway_mac_tracking: true }
  # option A instead of 4003:
  # 1420: { prefix: 10.9.90.0/24, gateway: 10.9.90.1, gateway_owner: ENT-FW,
  #         pool_start: 10.9.90.101, lab_pool_start: 10.9.90.151,
  #         vrf: FW-ZONE, gateway_mac_tracking: true }
  <PROD-VLAN-A>: { vsi: VPN-<PROD-VLAN-A>, prefix: <SUBNET-A>, gateway: <GW-A>,
                   gateway_owner: DC-FW, pool_start: <POOL-A>, vrf: DC-FW,
                   gateway_mac_tracking: true }
  <PROD-VLAN-B>: { vsi: VPN-<PROD-VLAN-B>, prefix: <SUBNET-B>, gateway: <GW-B>,
                   gateway_owner: DC-FW, pool_start: <POOL-B>, vrf: DC-FW,
                   gateway_mac_tracking: true }
# guard rails below the hard TRex limits (128 latency / 1023 flow-stat streams per instance)
budget: { max_latency_streams: 120, max_flowstat_streams: 600, max_pps: 50000,
          max_bps: 50000000, max_pps_per_fw_stream: 100 }

Every Leaf and Border Leaf AP carries all VLANs under test: the vlans_under_test anchor (&utv) is reused by every Leaf entry (*utv), while the Border Leaf entries list the same VLANs explicitly plus 4003, because YAML cannot splice an alias into a longer list. The loss-only coverage groups and the L3-DCFW class need the production VLANs at every AP. FWZ is not an attachment point of its own: it names the zone VLAN 4003 at K-BL, so ap: FWZ pins p6 to S-VLAN 3110 with VLAN 4003, and test commands can say --dst FWZ --dst-vlan 4003. The mac: of each lab port is its fixed identity: it must equal the src_mac of the same port in trex_lab.yaml and trex_lab_astf.yaml (B.1); the lab IP is derived from the pinned AP (lab_pool_start + index - 1). Rules the validator is meant to enforce (plan section 9.3): S-VLANs unique; AP indexes unique and between 1 and 49 (lab IPs use index + 50 in the lab pool; the 16-address pools cover indexes 1-16); lab-port MACs unique and equal to the src_mac entries of the TRex configuration files; every VLAN referenced by an AP exists; the monitoring pool and the lab pool (16 addresses each) stay inside the prefix and overlap neither each other nor the gateway; gateway_mac_tracking is set for every VLAN whose gateway is a firewall cluster - VLAN 4003 (1420 with option A) and the production VLANs - because the MAC of the active member changes on a failover. In addition, the VSI of every VLAN must exist on the Leaf of the AP - not every VSI exists on every Leaf; trexctl inventory render-leaf and the acceptance checks verify it. pg_ids are not part of the inventory: trexctl allocates them from fixed ranges - latency 1-127, loss-only 1000-1999, lab from 2000 - and keeps them stable in /data/trexctl/state/pgid-map.json (written by the trexctl service user, hence under /data with the other state files, not under /etc; include it in the nightly backup and copy it into the configuration repository when it changes - Chapter 18).

B.8 Default monitoring profile (planned format)¶

The profile implements the monitoring design: a limited set of latency streams (kind: latency, STLFlowLatencyStats, at most 128 per TRex instance) and a larger set of loss-only flow-stat streams (kind: flowstat, STLFlowStats, at most 1023 with --software). Explained in Chapter 16.

planned format · decision (production VLANs, firewall zone: option B shown) · source: plan v0.92 section 9.4 (groups and pairs); class: holds the tag value and the DSCP groups add path: (Chapter 18); budget as in Chapter 16

/etc/trexctl/profiles/monitoring-default.yaml - planned format

version: 1
name: monitoring-default
instance: mon
defaults: { kind: latency, pps: 200, size: 128, tos: 0, ttl: 64, bidir: true,
            dport_base: 32000, sport_base: 40000 }
# kind: latency   -> STLFlowLatencyStats: latency, jitter, loss, sequence errors
#                    (max 128 per instance); built with UDP checksum 0
# kind: flowstat  -> STLFlowStats: TX/RX packet counters per pg_id = loss only
#                    (max 1023 with --software)
# vlans: [...]        -> bridged: destination MAC = remote TRex client
# src_vlan / dst_vlan -> routed: destination MAC = gateway of src_vlan (EMU);
#                        destination = dst_vlan client at the SECOND AP of the pair;
#                        a firewall MAC is fixed in the stream until it is rebuilt
# pairs_from: [...]   -> re-uses the pair lists of the named groups
# bidir: true         -> two streams per pair, one pg_id per direction
# TRex sets bit 0 of the TOS byte of every statistics stream (0xB8 leaves as 0xB9)
groups:
  # ---------- latency streams: 74 streams, 10 400 pps ----------
  - name: l2-intra-rzk            # 5 pairs x 2 = 10
    kind: latency
    class: L2
    vlans: [1314]
    pps: 200
    pairs: [[K-L11, K-BL], [K-L12, K-BL], [K-L21, K-BL],
            [K-L31, K-BL], [K-L41, K-BL]]
  - name: l2-intra-rzc            # 6 pairs x 2 = 12
    kind: latency
    class: L2
    vlans: [1314]
    pps: 200
    pairs: [[C-L11, C-BL], [C-L12, C-BL], [C-L13, C-BL],
            [C-L21, C-BL], [C-L22, C-BL], [C-L23, C-BL]]
  - name: l2-inter-dc             # 6 pairs x 2 = 12
    kind: latency
    class: L2
    vlans: [1314]
    pps: 200
    pairs: [[K-L11, C-L11], [K-L12, C-L12], [K-L21, C-L21],
            [K-L31, C-L22], [K-L41, C-L23], [K-BL, C-BL]]
  - name: l3-bl                   # 4 x 2 = 8, routed on sw-rzk-BL
    kind: latency
    class: L3-BL
    src_vlan: 1314
    dst_vlan: 1315
    pps: 200
    pairs: [[K-L11, K-L21], [K-L31, K-BL], [C-L21, C-L22], [C-L13, C-BL]]
  - name: l3-dcfw                 # 4 x 2 = 8, routed by the DC-FW
    kind: latency
    class: L3-DCFW
    src_vlan: <PROD-VLAN-A>
    dst_vlan: <PROD-VLAN-B>
    pps: 50
    pairs: [[K-L11, K-L41], [K-L21, K-L31], [C-L12, C-L23], [C-L11, C-L22]]
  - name: l3-entfw                # 4 x 2 = 8, via ENT-FW to the zone
    kind: latency
    class: L3-ENTFW
    src_vlan: 1314
    dst_vlan: 4003                # zone client at the BL AP of the same DC
    pps: 50
    pairs: [[K-L11, K-BL], [K-BL, K-BL], [C-L21, C-BL], [C-BL, C-BL]]
    # option A: dst_vlan: 1420 and FW as the second AP of every pair
  - name: dscp-l2-interdc         # 4 TOS values x 2 directions = 8
    kind: latency
    class: DSCP
    path: L2
    vlans: [1314]
    pps: 100
    size: 256
    tos_set: [0x00, 0xB8, 0x88, 0x20] # BE (reference), EF, AF41, CS1
    pairs: [[K-L11, C-L21]]
  - name: dscp-entfw              # 4 TOS values x 2 directions = 8
    kind: latency
    class: DSCP
    path: L3-ENTFW
    src_vlan: 1314
    dst_vlan: 4003
    pps: 50
    size: 256
    tos_set: [0x00, 0xB8, 0x88, 0x20] # BE (reference), EF, AF41, CS1
    pairs: [[K-L11, K-BL]]
    # option A: dst_vlan: 1420, pairs: [[K-L11, FW]]
  # ---------- loss-only flow-stat streams: 102 streams, 10 200 pps ----------
  - name: l2-coverage-1315        # 17 pairs x 2 = 34
    kind: flowstat
    class: L2
    vlans: [1315]
    pps: 100
    pairs_from: [l2-intra-rzk, l2-intra-rzc, l2-inter-dc]
  - name: l2-coverage-prod        # 17 pairs x 2 x 2 VLANs = 68
    kind: flowstat
    class: L2
    vlans: [<PROD-VLAN-A>, <PROD-VLAN-B>]
    pps: 100
    pairs_from: [l2-intra-rzk, l2-intra-rzc, l2-inter-dc]
tags: { environment: prod, owner: netzwerk }

[K-BL, K-BL] in l3-entfw is not a typo: it sends from the VLAN 1314 client at K-BL to the VLAN 4003 zone client at the same AP - same TRex port and S-VLAN, different C-VLAN - and the firewall routes between the two. The loss-only groups are split by VLAN, not by pair set: pairs_from copies the 17 pairs of the three L2 latency groups.

Group	kind	Streams	pps per stream	Size (bytes)	pps total
l2-intra-rzk / -rzc	latency	10 + 12 = 22	200	128	4 400
l2-inter-dc	latency	12	200	128	2 400
l3-bl	latency	8	200	128	1 600
l3-dcfw	latency	8	50	128	400
l3-entfw	latency	8	50	128	400
dscp-l2-interdc / dscp-entfw	latency	8 + 8 = 16	100 / 50	256	800 + 400 = 1 200
Latency streams		74 (limit 128, budget 120)			10 400
l2-coverage-1315	flowstat	34 (17 pairs x 2, VLAN 1315)	100	128	3 400
l2-coverage-prod	flowstat	68 (17 pairs x 2 x 2 production VLANs)	100	128	6 800
Flow-stat streams		102 (limit 1023, budget 600)			10 200
Total		176			20 600 pps, about 22 Mbit/s at Layer 2, about 26 Mbit/s on the wire

The totals: latency 22 x 200 + 12 x 200 + 8 x 200 + 8 x 50 + 8 x 50 + 8 x 100 + 8 x 50 = 10 400 pps; flow-stat (34 + 68) x 100 = 10 200 pps. At Layer 2 (without FCS) 19 400 pps of 128-byte frames and 1 200 pps of 256-byte frames make about 22.3 Mbit/s; with FCS, preamble and inter-frame gap (24 bytes more per frame) about 26.3 Mbit/s on the wire.

The resolution follows from the rate: 5 ms for the 200 pps latency streams, 10 ms for the flow-stat streams and the L2 DSCP streams, 20 ms for the firewall classes. 54 of the 128 latency streams remain free for ad-hoc latency tests and growth. Each DSCP group contains a BE reference (TOS 0x00): every class is compared with a stream that is identical except for the TOS byte - same pair, 256 bytes, same rate. On the wire TRex sets bit 0 of the TOS byte of every statistics stream (0x01, 0xB9, 0x89, 0x21); the DSCP value is unchanged.

Streams with a firewall destination MAC

Both directions of l3-dcfw and the direction from the zone clients (VLAN 4003 at K-BL and C-BL) of l3-entfw and dscp-entfw are sent to a firewall MAC. The clusters answered with member MACs in February 2025, so after a failover these streams must be rebuilt with the newly resolved MAC. The planned trexctl gateway-MAC tracker polls dgw.rmac every 10 s for every VLAN with gateway_mac_tracking: true, rebuilds the affected streams and writes the event gateway_mac_changed. How fast EMU learns the new MAC depends on how ClusterXL announces it: a gratuitous ARP request is learned at once, a broadcast ARP reply is ignored and the next periodic query (every 60 s by default) is needed. The plan's target of 15 s therefore needs a shorter ARP timer for the firewall VLANs or a forced query confirm on device. Natively, re-create the groups as in incident I3 of Chapter 23. A ClusterXL virtual MAC would avoid the rebuild decision (B.6).

Until trexctl exists, every line of this table is built with the Python API of Appendix A.4: one STLStream with STLFlowLatencyStats(pg_id) or STLFlowStats(pg_id) per stream and direction, latency streams with UDP checksum 0 (UDP(..., chksum=0)).

B.9 InfluxDB installation and set-up¶

verified commands and keys (InfluxDB OSS v2 documentation: "Install InfluxDB", "Configuration options", influx setup, influx bucket, influx auth create; the InfluxData RPM repository lists influxdb2 2.9.1 and influxdb2-cli 2.8.0) · confirm on device paths and sizes · source: plan v0.92 section 10.1. Explained in Chapter 8.

InfluxDB OSS 2.x on Rocky Linux / RHEL 9 (as root)

# 1. repository - official procedure: check the key fingerprint first
curl --silent --location -O https://repos.influxdata.com/influxdata-archive.key
gpg --show-keys --with-fingerprint --with-colons ./influxdata-archive.key 2>&1 \
  | grep -q '^fpr:\+24C975CBA61A024EE1B631787C3D57159FC2F927:$' \
  && cat influxdata-archive.key | gpg --dearmor \
     | tee /etc/pki/rpm-gpg/RPM-GPG-KEY-influxdata > /dev/null
cat > /etc/yum.repos.d/influxdata.repo <<'EOF'
[influxdata]
name = InfluxData Repository - Stable
baseurl = https://repos.influxdata.com/stable/$basearch/main
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-influxdata
EOF

# 2. packages - stay on 2.x (InfluxDB 3 is a different product and engine)
dnf -y install influxdb2-2.9.1 influxdb2-cli

# 3. data on /data (package defaults: data in /var/lib/influxdb,
#    configuration file /etc/influxdb/config.toml)
mkdir -p /data/influxdb && chown influxdb:influxdb /data/influxdb
cat > /etc/influxdb/config.toml <<'EOF'
bolt-path = "/data/influxdb/influxd.bolt"
engine-path = "/data/influxdb/engine"
http-bind-address = "127.0.0.1:8086"
query-memory-bytes = 2147483648
storage-cache-max-memory-size = 2147483648
EOF
systemctl enable --now influxdb

# 4. initial set-up: organisation lvr, bucket trex (90 days raw data)
influx setup --org lvr --bucket trex --retention 90d --username influxadmin \
    --password '<STRONG-PASSWORD>' --force
influx bucket create --org lvr --name trex_longterm --retention 730d

# 5. tokens with minimal rights (bucket IDs from the list)
TREX_ID=$(influx bucket list --org lvr --name trex --hide-headers \
          | awk '{print $1}')
LT_ID=$(influx bucket list --org lvr --name trex_longterm --hide-headers \
        | awk '{print $1}')
influx auth create --org lvr --description "trexctl exporter (write trex)" \
    --write-bucket "$TREX_ID"
influx auth create --org lvr --description "grafana (read)" \
    --read-bucket "$TREX_ID" --read-bucket "$LT_ID"

Element	Why
`http-bind-address = "127.0.0.1:8086"`	only local clients (exporter, Grafana, CLI) reach InfluxDB
`bolt-path`, `engine-path` on `/data`	metadata and time-series data on the data RAID; the SQLite file follows the bolt path directory
two buckets	`trex`: raw 2-second data for 90 days; `trex_longterm`: 1-minute aggregates for 730 days (B.10)
separate write and read tokens	the exporter can only write `trex`, Grafana can only read

Tokens are shown once

Since InfluxDB OSS 2.9.0 API tokens are stored as hashes: copy each token from the output of influx auth create into the password safe immediately. influx setup stores the operator token in the CLI connection configuration of the user who ran it. A backup of a 2.9 instance contains no plaintext operator token, and a full restore needs influx restore --operator-token (InfluxDB documentation, "Manage API tokens - Token hashing" and "Back up data").

B.10 Flux downsampling task¶

Raw data stays 90 days in trex; the task writes 1-minute aggregates into trex_longterm. Field names follow the data model of Chapter 18: lat_max_us is the per-interval maximum that the exporter derives from the histogram deltas; lat_last_max_us holds TRex's last_max, the maximum of the last 0.5-second server window only - it is kept for reference and is never used as the interval maximum. Each pipeline ends with a named yield() because a Flux script with several outputs needs unique result names (Flux documentation, yield()).

confirm on device (field names depend on the exporter implementation) · Flux functions verified (Flux documentation: aggregateWindow(), to(), yield(); task options) · source: plan v0.92 section 10.2, field names as in the data model of plan section 9.7 and Chapter 18

/opt/trexctl/influx/downsample.flux

option task = {name: "trex_downsample_1m", every: 10m, offset: 1m}

src = from(bucket: "trex")
  |> range(start: -20m)
  |> filter(fn: (r) => r._measurement == "trex_stream")

src
  |> filter(fn: (r) => r._field == "lat_avg_us" or r._field == "jitter_us"
                       or r._field == "tx_pps" or r._field == "rx_pps")
  |> aggregateWindow(every: 1m, fn: mean, createEmpty: false)
  |> to(bucket: "trex_longterm", org: "lvr")
  |> yield(name: "mean")

src
  |> filter(fn: (r) => r._field == "lat_max_us")
  |> aggregateWindow(every: 1m, fn: max, createEmpty: false)
  |> to(bucket: "trex_longterm", org: "lvr")
  |> yield(name: "max")

src
  |> filter(fn: (r) => r._field == "drop_pkts" or r._field =~ /^hist_/)
  |> aggregateWindow(every: 1m, fn: sum, createEmpty: false)
  |> to(bucket: "trex_longterm", org: "lvr")
  |> yield(name: "sum")

Create and check the task

influx task create --org lvr -f /opt/trexctl/influx/downsample.flux
influx task list --org lvr
influx task run list --task-id <TASK-ID>
influx task log list --task-id <TASK-ID>

Element	Why
`every: 10m`	tasks run at multiples of the interval, so every range starts on a full minute and no 1-minute window is cut
`offset: 1m`	delays the run for late points; the time range stays relative to the scheduled time (task options)
`range(start: -20m)`	the last two intervals are recomputed; identical timestamps overwrite the earlier result
mean / max / sum	averages for rates, latency and jitter; maxima for the interval maximum `lat_max_us`; sums for the per-interval counts (`drop_pkts`, `hist_*`), so the 1-minute histogram keeps the latency distribution for long-term percentiles. The reference field `lat_last_max_us` is not copied: the 1-minute maximum of `lat_max_us` already covers every packet of the minute

B.11 Grafana data sources¶

verified provisioning format (Grafana documentation: "Configure the InfluxDB data source - Provisioning examples, InfluxDB 2.x for Flux"; "Provision Grafana") · source: plan v0.92 section 10.3. Explained in Chapter 9 and Chapter 19.

/etc/grafana/provisioning/datasources/influx.yaml

apiVersion: 1
datasources:
  - name: InfluxDB-trex
    uid: influx-trex                  # (1)!
    type: influxdb
    access: proxy
    url: http://127.0.0.1:8086
    isDefault: true
    jsonData:
      version: Flux
      organization: lvr
      defaultBucket: trex
    secureJsonData:
      token: "<GRAFANA-READ-TOKEN>"   # (2)!
  - name: InfluxDB-trex-longterm
    uid: influx-trex-longterm
    type: influxdb
    access: proxy
    url: http://127.0.0.1:8086
    jsonData:
      version: Flux
      organization: lvr
      defaultBucket: trex_longterm
    secureJsonData:
      token: "<GRAFANA-READ-TOKEN>"

A fixed uid lets provisioned dashboards reference the data source independently of its display name.
The read token from B.9. Grafana also accepts $ENV_VAR_NAME here (environment variable lookup in provisioning files). Protect the file (owner root, group grafana, mode 0640). tlsSkipVerify is not needed: the connection is plain HTTP on the loopback interface.

/etc/grafana/grafana.ini - changed settings

[server]
protocol = https
http_port = 443
cert_file = /etc/grafana/tls.crt
cert_key = /etc/grafana/tls.key
root_url = https://trex-rzk-01.<DOMAIN>/

[dashboards]
min_refresh_interval = 1s

/etc/systemd/system/grafana-server.service.d/bind443.conf - lets the grafana user bind port 443

[Service]
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE

verified keys of grafana.ini (Grafana documentation, "Configure Grafana": [server] protocol, http_port, cert_file, cert_key, root_url; [dashboards] min_refresh_interval, default 5 s) · confirm on device certificate paths and the systemd drop-in. Grafana is installed from the repository https://rpm.grafana.com with dnf install grafana (Grafana documentation, "Install Grafana on RHEL or Fedora"); pin the version (12.4.x or 13.x) according to the LVR patch policy.