AI DEPLOYMENT / DISTRIBUTED INFERENCE

Run DeepSeek V4 Flash on Four GEEKOM A9 Mega PCs

Connect four A9 Mega systems over USB4, split the model layers across them with DS4, and test a local chat API.

4 × GEEKOM A9 Mega128 GB per nodeUbuntu 26.04 LTSUSB4 networking

Reference configuration. The hardware is four GEEKOM A9 Mega PCs with 128 GB each. The software stack and commands below are a starting point. Confirm the BIOS, kernel, packages, DS4 commit and a successful four-node request on your systems before relying on the setup.

Before you start

This guide uses pipeline parallelism: each PC runs a different range of model layers. PC1 coordinates the pipeline and serves the API. The setup distributes the GPU workload across the machines; it does not guarantee higher token generation speed.

  • Four GEEKOM A9 Mega PCs: AMD Ryzen AI Max+ 395, Radeon 8060S and 128 GB LPDDR5X unified memory per PC.
  • Three USB4 or Thunderbolt data cables that support host-to-host networking. Use the rear USB4 ports; a USB-C charging cable or a front data-only port is not a substitute.
  • Local NVMe storage: the GGUF is 86.7 GB (about 80.8 GiB) per node. Plan at least 100 GiB free per PC for the model and working files, in addition to space for Ubuntu.
  • Local console access on every PC for network changes, plus a separate Ethernet or Wi-Fi connection for software downloads. Keep the USB4 cluster links on a dedicated subnet.

Check your A9 Mega specifications

Wire the four PCs

Connect PC1 ↔ PC2, PC2 ↔ PC3 and PC3 ↔ PC4. Do not connect PC4 back to PC1. Leave the regular Ethernet or Wi-Fi connection separate from the USB4 bridge.

USB4 chain and model layersPC1 connects to PC2, PC2 to PC3, and PC3 to PC4. PC2 and PC3 bridge both USB4 links. Layer computation follows PC1 to PC4.USB4 · 1USB4 · 2USB4 · 3PC1Coordinator192.168.10.10Endpoint0:10PC2Worker 1192.168.10.20Bridge11:20PC3Worker 2192.168.10.30Bridge21:30PC4Worker 3192.168.10.40Endpoint31:outputUSB4 chain and model layersPC1 connects to PC2, PC2 to PC3, and PC3 to PC4. PC2 and PC3 bridge both USB4 links. Layer computation follows PC1 to PC4.USB4 · 1USB4 · 2USB4 · 3PC1Coordinator192.168.10.100:10EndpointPC2Worker 1192.168.10.2011:20BridgePC3Worker 2192.168.10.3021:30BridgePC4Worker 3192.168.10.4031:outputEndpoint
Three separate USB4 cables form a single chain. The layer ranges shown are the example used throughout this guide.
NodeCluster IPNetwork roleDS4 roleLayers
PC1192.168.10.10One direct linkCoordinator0:10
PC2192.168.10.20Two links via br0Worker 111:20
PC3192.168.10.30Two links via br0Worker 221:30
PC4192.168.10.40One direct linkWorker 331:output
Rear USB4 port concept
Identify the two rear USB4 portsConcept diagram of two USB4 Type-C ports on the rear of a middle node. Each goes to a different neighbor. REAR USB4 PAIR One neighbor Other neighbor
Concept diagram for PC2 and PC3. Use both rear USB4 ports, one for each adjacent PC. This drawing does not show the exact rear panel layout.

Configure the cluster network

The example subnet is 192.168.10.0/24. If your normal LAN or VPN already uses it, pick a different private subnet and update every example IP, including the DS4 commands. Keep the existing default route on your regular connection.

Identify the USB4 interfaces

At each PC, run the commands below. If needed, disconnect and reconnect one USB4 cable at a time to map the interfaces. Linux names can vary; substitute the actual names in the YAML, including the bridge interface list.

Bash
sudo modprobe thunderbolt-net
ip -br link
ip -br addr

Back up Netplan and create the file

Run on the local console of each PC. Use the file only if that name is unused, and check existing Netplan or NetworkManager settings for conflicting definitions of these USB4 interfaces. Paste just the YAML for the current PC.

Bash
sudo cp -a /etc/netplan "/root/netplan-before-a9-$(date +%Y%m%d-%H%M%S)"
sudoedit /etc/netplan/90-a9-cluster.yaml
PC1 · 192.168.10.10
YAML
network:
  version: 2
  ethernets:
    thunderbolt0:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
      addresses: [192.168.10.10/24]
PC2 · 192.168.10.20
YAML
network:
  version: 2
  ethernets:
    thunderbolt0:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
    thunderbolt1:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
  bridges:
    br0:
      renderer: networkd
      interfaces: [thunderbolt0, thunderbolt1]
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
      addresses: [192.168.10.20/24]
      parameters:
        stp: true
PC3 · 192.168.10.30
YAML
network:
  version: 2
  ethernets:
    thunderbolt0:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
    thunderbolt1:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
  bridges:
    br0:
      renderer: networkd
      interfaces: [thunderbolt0, thunderbolt1]
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
      addresses: [192.168.10.30/24]
      parameters:
        stp: true
PC4 · 192.168.10.40
YAML
network:
  version: 2
  ethernets:
    thunderbolt0:
      renderer: networkd
      dhcp4: false
      dhcp6: false
      link-local: []
      optional: true
      mtu: 1500
      addresses: [192.168.10.40/24]

Check, try and confirm the configuration

Confirm netplan try only when the cluster IP and regular network both work. Netplan rollback has caveats with bridges; if a trial fails, check the restored state at the local console. The examples use MTU 1500 on all links. Leave jumbo frames for a separately tested configuration.

Bash
sudo chown root:root /etc/netplan/90-a9-cluster.yaml
sudo chmod 600 /etc/netplan/90-a9-cluster.yaml
sudo netplan generate
sudo netplan try

Test the full chain

From PC1, ping PC2, PC3 and PC4. Repeat from the Workers to PC1 and to neighboring Workers, since pipeline stages exchange data directly. On PC2 and PC3, bridge link should show both USB4 links as members of br0.

Bash
ping -c 3 192.168.10.20
ping -c 3 192.168.10.30
ping -c 3 192.168.10.40

Prepare Ubuntu and ROCm

Install or confirm Ubuntu

Use Ubuntu 26.04 LTS Desktop (AMD64) and record the point release and kernel actually installed on all four PCs. If installing from scratch, verify the ISO checksum, use the model’s UEFI boot menu and back up the target drive before choosing an erase option. For an existing working installation, skip the reinstall.

Ubuntu 26.04 LTS downloads and checksums · Ubuntu Desktop installation documentation

Install the GPU runtime and build dependencies

Use one package source consistently. The block below follows the DS4 upstream native Ubuntu 26.04 example using Ubuntu packages. Run it on all four PCs. If these packages are unavailable or your DS4 commit has different prerequisites, use its Strix Halo documentation and a matching AMD/Ubuntu installation route. Do not add a focal repository or mix development headers from different ROCm versions.

Bash
sudo apt update
sudo apt install build-essential git curl python3 hipcc rocminfo rocm-smi \
  libamdhip64-dev libhipblas-dev libhipblaslt-dev librocblas-dev \
  librocwmma-dev libhipcub-dev
sudo usermod -aG render,video "$USER"

Sign out and back in after the group change. Check that rocminfo can see gfx1151 as a normal user before continuing. Record the actual package versions and keep all four hosts aligned.

Bash
cat /etc/os-release
uname -r
id -nG
ls -l /dev/kfd
rocminfo | grep -E 'gfx1151|Radeon 8060S'
hipcc --version
dpkg-query -W hipcc libamdhip64-dev libhipblas-dev
GPU-visible memory and firmware

Start with the current firmware and kernel settings. The 128 GB is shared memory, and the GPU cannot necessarily address all of it. Only tune if a confirmed GPU allocation failure persists after verifying the runtime, file and small context. Check the DS4 and AMD guidance for the installed kernel; preserve existing boot options, leave memory for Ubuntu, record the previous settings, and change one setting at a time.

AMD Ryzen AI Max memory and kernel guidance · DS4 for AMD Strix Halo

Build DS4 and download the model

Clone the upstream antirez/ds4 repository on PC1 and record its full commit hash. Use the exact same commit on PC2–PC4. These are first-install examples; for an existing checkout, check its status and use the intended commit instead of cloning over it.

PC1: choose and record the commit

Bash
mkdir -p "$HOME/a9-cluster"
cd "$HOME/a9-cluster" || exit
git clone https://github.com/antirez/ds4.git
git -C ds4 rev-parse HEAD

PC2–PC4: check out the same commit

Bash
mkdir -p "$HOME/a9-cluster"
cd "$HOME/a9-cluster" || exit
git clone https://github.com/antirez/ds4.git
read -r -p 'Paste the full PC1 DS4 commit: ' commit
git -C ds4 checkout --detach "$commit" &&
  git -C ds4 rev-parse HEAD

All four PCs: build and check the command-line options

Bash
cd "$HOME/a9-cluster/ds4" &&
  make strix-halo &&
  ./ds4 --help &&
  ./ds4-server --help

Before starting, confirm that both binaries built, the ROCm backend is available, and the selected commit supports pipeline options --role, --layers, --listen and --coordinator where applicable. The Coordinator also needs the server options --host and --port.

Download and verify the same GGUF on all four PCs

This example uses the original guide’s DeepSeek V4 Flash mixed Q2 file, quantized by jedisct1 for DS4. Download once and copy it to the other PCs if convenient. On every node, verify the published SHA-256. If the file is rejected by your chosen DS4 commit, resolve that mismatch against its model documentation before proceeding.

Bash
mkdir -p "$HOME/a9-cluster/models"
cd "$HOME/a9-cluster/models" || exit
df -h .
model='DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf'
curl -fL --continue-at - --output "$model.part" \
  "https://huggingface.co/jedisct1/DeepSeek-V4-Flash-imatrix-aligned/resolve/main/$model" &&
  printf '%s  %s\n' '94c95d56b83476547d55219a062a1095c523318d3fba6bd80e4d413687689100' "$model.part" | sha256sum --check - &&
  mv -- "$model.part" "$model"
If you copied the GGUF from another PC
Bash
cd "$HOME/a9-cluster/models" || exit
model='DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf'
printf '%s  %s\n' '94c95d56b83476547d55219a062a1095c523318d3fba6bd80e4d413687689100' "$model" | sha256sum --check -

Start the four nodes

Open a terminal on each PC. Start PC4, PC3 and PC2, then start PC1. Keep the processes in the foreground for the first run. Use the logs to see which Workers connect and which ranges register; there is no fixed wait between commands. All nodes start at --ctx 4096 to limit memory use.

Start in this order

1. PC42. PC33. PC24. PC1
1. PC4 · Worker 3 · 31:output
Bash
cd "$HOME/a9-cluster/ds4" || exit
model="$HOME/a9-cluster/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf"
./ds4 --rocm -m "$model" --ctx 4096 \
  --role worker --layers 31:output \
  --listen 192.168.10.40 1235 --coordinator 192.168.10.10 1234
2. PC3 · Worker 2 · 21:30
Bash
cd "$HOME/a9-cluster/ds4" || exit
model="$HOME/a9-cluster/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf"
./ds4 --rocm -m "$model" --ctx 4096 \
  --role worker --layers 21:30 \
  --listen 192.168.10.30 1235 --coordinator 192.168.10.10 1234
3. PC2 · Worker 1 · 11:20
Bash
cd "$HOME/a9-cluster/ds4" || exit
model="$HOME/a9-cluster/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf"
./ds4 --rocm -m "$model" --ctx 4096 \
  --role worker --layers 11:20 \
  --listen 192.168.10.20 1235 --coordinator 192.168.10.10 1234
4. PC1 · Coordinator · 0:10
Bash
cd "$HOME/a9-cluster/ds4" || exit
model="$HOME/a9-cluster/models/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-aligned.gguf"
./ds4-server --rocm -m "$model" --ctx 4096 \
  --role coordinator --layers 0:10 \
  --listen 192.168.10.10 1234 --host 127.0.0.1 --port 8080 --debug

The API listens on PC1 at 127.0.0.1:8080 for the first test. The Workers register with PC1 on TCP 1234. The example fixes Worker data listeners to TCP 1235 on their individual cluster IPs. Keep the DS4 data plane on a trusted network; it has no built-in transport authentication or encryption.

If a firewall is active, allow only the required traffic on the USB4 interface or bridge: PC2–PC4 to PC1 TCP 1234, and cluster peers to Worker TCP 1235. Keep the firewall enabled. Do not forward these ports from the internet.

Verify the chat API

Run the following from a second terminal on PC1. /v1/models checks the API listener. A chat reply with a nonempty choices[0].message.content, together with all three registered Worker ranges in the Coordinator log, verifies the full route.

Bash
curl -fsS http://127.0.0.1:8080/v1/models |
  python3 -m json.tool

curl --fail-with-body -sS http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Explain local AI in one sentence."}],"max_tokens":64,"think":false,"stream":false}' |
  python3 -m json.tool

A successful model list alone does not verify distributed inference. Keep the terminal logs from all four PCs when checking the result.

Optional: allow a trusted cluster client

After local testing, restart PC1 with --host 192.168.10.10 instead of the loopback address if another node must call the API. Permit TCP 8080 only from trusted clients on the chosen interface. Use an authenticated, encrypted proxy or a VPN if the API must cross network boundaries.

Troubleshoot and recover

No USB4 interface

Use the rear USB4 ports and a known-good data cable. Replug one link at a time, confirm thunderbolt-net is loaded and check ip -br link.

PC4 is unreachable

Check each cable in order, the subnet and MTU, and the two members of br0 on PC2 and PC3. Make sure the cluster subnet does not overlap your LAN or VPN.

A Worker does not register or the route is incomplete

Match the commit, GGUF checksum, context size and layer table. Check bind addresses, local firewall scope and the Worker data port in the logs. Do not send HTTP requests to TCP 1234.

The GPU is missing or runs out of memory

Verify gfx1151, /dev/kfd access and group membership. Close other GPU workloads and retry the baseline before changing firmware or boot parameters.

The chat API responds with an error

Read the response body and Coordinator log. Confirm the three Workers are connected and all ranges are present, and use the model alias reported by /v1/models.

Stop or roll back

Stop the foreground processes with Ctrl+C. If you added only the new YAML file, move it out of /etc/netplan and run sudo netplan apply at the local console. Restore the earlier settings from your backup if you also modified existing files.

References