GMKtec EVO-X2 Mini PC Review: 128GB AMD Ryzen AI Max+ for Local LLMs vs Mac Mini & Nvidia DGX Spark

Alex Ziskind

Introduction and Device Overview [00:00]

• The GMKtec EVO-X2 is presented as a mini PC alternative while waiting for products like the Nvidia DGX Spark. [00:06]
• It features 128GB of RAM shared with the GPU, designed for local LLMs and machine learning. [00:12]
• The price is noted as significantly less than the DGX Spark. [00:22]
• The device is currently in high demand due to its new AMD chip. [00:42]

Hardware Specifications and Ports [00:48]

• The EVO-X2 is described as a beast compared to GMKtec's usual lineup. [00:47]
• It is loaded with ports: [00:51]
  • Two USB 4 ports
  • Five USB-A ports
  • SD card reader
  • HDMI port
  • DisplayPort
  • Two headset ports
  • A 2.5 Gb Ethernet port (corrected in description) [01:00]

    

• The core component is the AMD Ryzen AI Max+ 395 chip: [01:05]
  • 16 cores, 32 threads
  • Up to 5.1 GHz clock speed
  • Integrated GPU (Radeon 8060S) claimed to rival an RTX 4060. [01:16]
• Memory: [01:24]
  • Available in 64GB and 128GB versions.
  • LPDDR5X mobile memory at 8000 MHz. [01:30]
• Storage: [01:33]
  • Dual PCIe Gen 4 SSD slots.
  • Upgradeable up to 16 TB. [01:36]
• Networking: [01:41]
  • Wi-Fi 7 support.

Initial LLM Performance Testing (LM Studio) [02:46]

• A baseline test is performed using LM Studio. [02:46]
• Test 1: Quinn 32 Billion Q4 Model [02:55]
  • Model size: 18.4 GB on disk.
  • LM Studio recommended offloading 64/64 layers to the GPU. [03:02]
  • The model is loaded into system memory then copied to dedicated GPU memory due to static partitioning. [03:13]
  • Dedicated GPU memory usage: 18.4 GB. [03:31]
  • Prompt: "hi" (one token for fastest speed baseline) [03:34]
  • Result: 10.58 tokens per second. [03:58]
  • Power consumption during this test: Around 170-174 watts. [05:06]

    

• Test 2: Llama 3.3 70 Billion Q4 Model [04:12]
  • Model size: 39 GB on disk. [04:15]
  • LM Studio recommended offloading 75/80 layers; tested by offloading all 80 layers. [04:21]
  • Offloading layers to CPU significantly slows down performance. [04:29]
  • Dedicated GPU memory usage: 43 GB. [04:39]
  • Prompt: "hi" [04:48]
  • Result: 5.1 tokens per second. [04:53]
  • GPU is fully utilized during generation. [04:57]
  • This speed is considered a bit slow for practical use. [14:55]

Memory Bandwidth Importance and Benchmarks [05:14]

• Memory bandwidth is identified as the most important factor for LLM speed. [05:16]
• The Stream benchmark is used to measure memory bandwidth. [05:28]

  

• M4 Mac Mini (Base Model) Stream Benchmark: [05:43]
  • Apple advertises 120 GB/s. [05:45]
  • Measured result (Triad number): 96 GB/s. [05:56]

    

• M4 Pro Mac Mini Stream Benchmark: [13:08]
  • Apple advertises 273 GB/s. [11:58]
  • Measured result: 209 GB/s. [13:08]
• GMKtec EVO-X2 Stream Benchmark: [13:13]
  • Published memory bandwidth: 256 GB/s. [12:14]
  • Measured result (Triad number): 120 GB/s. [13:19]
  • This measured result is much lower than expected/published, potentially indicating an issue with the benchmark or drivers. [13:22]

    

LLM Performance Comparisons (GMKtec vs M4 vs M4 Pro) [06:09]

• Model: DeepSeek R1 Distill Qwen 7 Billion Q4 [06:09]

  • Model size: ~4 GB on disk. [06:14]
  • Runs fully on GPU. [06:17]
  • M4 Mac Mini: 21 tokens per second (with short prompt). [06:22]
    • Power usage: 31 watts. [06:34]
    • GPU fully utilized. [06:42]

      

  • GMKtec EVO-X2: 43 tokens per second (with short prompt). [06:56]

    

  • M4 Pro Mac Mini: 44 tokens per second (with short prompt). [11:30]

    

• Llama Bench Benchmark (DeepSeek R1 Distill Qwen 7 Billion Q4): [09:48, 10:37]

  • Measures Prompt Processing (PP) and Token Generation (TG).
  • GMKtec EVO-X2 (Vulcan backend): [09:54]
    • PP512: 935.75 tokens per second. [10:06]
    • TG128: 46.25 tokens per second. [10:13]

      

  • M4 Mac Mini: [10:37]
    • PP512: 228.72 tokens per second. [10:43]
    • TG128: 21.59 tokens per second. [10:47]

      

• Comparison Summary (based on benchmarks): [11:35]

  • The M4 Pro doubles the performance of the base M4, primarily due to increased memory bandwidth (273 GB/s vs 120 GB/s). [11:51]
  • The M4 Max and M3 Ultra offer even higher bandwidths (540 GB/s and 819 GB/s respectively). [12:01]
  • The GMKtec EVO-X2's AMD chip (256 GB/s published) performs comparably to the M4 Pro in LLM token generation despite lower measured Stream bandwidth, and significantly outperforms the base M4. [12:19, 13:51]

    

Technical Aspects: Llama CPP, Vulcan, and Static Partitioning [08:23, 16:00]

• LM Studio runs on Llama CPP, which can be compiled with different backends (Rockom for AMD, Vulcan for cross-platform, CPU only). [08:26]
• Vulcan is a cross-platform library but has some overhead compared to more direct approaches like Rockom. [08:57]
• The EVO-X2 uses static memory partitioning, where memory is divided into fixed blocks for system and GPU before startup. [16:34]
  • Advantages: Easier setup, fewer system resources used, stability. [16:43]
  • Disadvantages: Wastes unused memory in blocks, limits the number of programs that can run simultaneously. [17:06]

    

• This differs from Apple's unified memory architecture, where memory is dynamically shared and accessed by all components on the chip. [02:09, 16:07]

  

AMD Software Support Challenges (Rockom) [17:26]

• Rockom is AMD's open-source software stack for accelerated computing. [17:30]
• Official Rockom support for the Ryzen AI Max+ chip is not yet stable, requiring specific OS and kernel versions and potentially involving unstable patched versions of software like Olama or Llama CPP. [17:33, 17:59]
• AMD has stated they are working on broader Rockom support for more chips, including this one, and are paying more attention to the LLM community. [17:36, 17:43]

  

• Future Rockom support is expected to significantly improve performance on this hardware. [17:28, 18:32]

Overall Performance Summary & Chart Analysis [14:13, 14:58]

• The EVO-X2 shows good performance numbers for various models: [14:13]
  • Gemma 3 1B: ~160 tokens/sec. [14:15]
  • Llama 3.2 1B Instruct: 130 tokens/sec. [14:20]
  • Gemma 3 4B: 66 tokens/sec. [14:24]
  • Deep Seek Distill 7B: 44 tokens/sec. [14:27]
  • Gemma 3 12B: 23.9 tokens/sec. [14:32] (More useful models)
  • 14B models: 23.7 tokens/sec. [14:37]
  • Quen 2.5 32B: 10.8 tokens/sec. [14:41] (Reasonable but not ideal speed)
  • Llama 3.3 70B: 5 tokens/sec. [14:48] (Too slow for practical use)

    

• Comparing the desktop (EVO-X2) to the laptop (Asus Flow Z13) with the same chip, the desktop version is slightly faster across most models, likely due to better thermals. [14:58]

  

• The EVO-X2 (AMD chip) performs better than the Mac Mini M4 and comparably to the Mac Mini M4 Pro for a 12 billion parameter model (23 t/s vs 49 t/s on M4 Max - correction: this comparison is M4 Max not M4 Pro). [15:48]

Price, Value, and Conclusion [18:41]

• Mini PCs, including the EVO-X2, are becoming more powerful with shrinking chip sizes. [18:46]
• The EVO-X2 is priced higher than average mini PCs: [18:59]
  • 64GB version: $1499
  • 128GB version: $2000 [19:04]
• Considering its performance and capabilities for LLMs, especially the 128GB model, the price is considered reasonable and offers significant value as a powerful machine in a small package. [19:11]
• Compared to the M4 Mac Mini, the EVO-X2 is more powerful for LLMs due to memory bandwidth and core count. [19:19]
• A 64GB M4 Pro Mac Mini costs $2399, making the 128GB EVO-X2 a more cost-effective option for high-memory LLM tasks. [19:35]

  

• The EVO-X2 is concluded to be a good sidekick machine for AI/dev work, with potential for further performance improvements with future software support. [17:22]