The MTP draft head is recurrent, so a single-layer MTP checkpoint emits N speculative tokens per step. The spec-N curve (N=1/2/4) is the real story: N=4 wins throughput at 204.6 t/s single-stream decode (+41% vs the community 145 claim) even though per-token acceptance falls with N (92.6% → 84.2% → 72.7% → 68.9% by draft position). N=1 reaches 97.1% acceptance — but costs 30% throughput. The right objective for a single-layer draft head is throughput, not acceptance %. A paired, alternating power A/B also killed the 'boost prefill to 230W' hypothesis: 150W vs 230W prefill is flat at ±0.2%.
Issue
Two assumptions blocked progress: (1) 'num_speculative_tokens=2 clamps to 1' — false, the MTP module is recurrent (spec_step_idx % num_mtp_layers), so a single layer emits N draft tokens per step. (2) 'prefill scales with power' — a claim from unpaired runs that turned out to be prefix-cache contamination, not a real power effect.
Solution
Ran a clean spec-token sweep (MTP1/2/4) with an honest cold-prefill harness (unique random prefix per call so prefix caching never fires), read acceptance directly from the vllm:spec_decode_* counters, then ran a paired alternating 150W-vs-230W power A/B to settle the power question rigorously.
local-aivllmxpuintel-arc
A phase-separated vLLM XPU study on Intel Arc Pro B70: cold input, p512 and p8192 decode, the p9445 control, and exact-128K decode across no-spec, MTP1, MTP2, and MTP4.
Issue
Older public surfaces mixed short-generation peaks, cold input rate, exact-128K completion, resident-session latency, and historical engine grids. Those metrics answer different questions.
Solution
The current recipe pins a public vLLM XPU image by digest, applies two patches in order, calibrates exact prompt shapes, and measures no-spec, MTP1, MTP2, and MTP4 in separate C1 input and output phases.
local-aivllmllama.cppsycl
Community threads claim vLLM XPU gives 10x the speed of llama.cpp on the B70 (11,000 t/s prefill, 150 t/s decode). I tested the full path — image, FP8, MXFP4, self-built native checkpoints, seven engine patches. The measured truth before MTP speculative decoding: llama.cpp wins single-stream by 6x, and the '150 t/s' claim turns out to be vLLM at 16 concurrent users. (See Phase 2 for the speculative decoding breakthrough).
Issue
The vLLM path has three gates, and each one turned out to be real: (1) the FP8 checkpoint for this model is 37.5 GB — it does not fit 32 GB of VRAM, so FP8 is off the table for 35B; (2) the MXFP4 (Intel 4-bit) checkpoints that exist publicly are in the compressed-tensors layout, which vLLM's XPU build rejects; (3) the prebuilt Intel images have an MXFP4 MoE path written for gpt-oss — the Qwen loader is broken in seven separate places, from tensor shapes to activation gating to hybrid-model page sizes.
Solution
I built the native-format MXFP4 checkpoint myself (fused 256 experts per layer into the w13/w2 layout, verified to 8.8e-5 MSE against BF16 ground truth), moved to the newer intel/vllm:0.17.0-xpu image, and patched the seven engine bugs in-container — 2D per-expert loader support, missing scale-key mapping, the silu-vs-swiglu_oai activation gate (the XPU kernel supports silu; the Python gate doesn't), CUDA-only device contexts in the linear-attention path, a contiguity check, and the hybrid block-size alignment that produced a page size the XPU flash-attention kernel rejects. The model then served and generated correct output.
local-aivllmllama.cppsycl
A single llama.cpp upgrade — oneDNN SDPA extended to quantized KV (#25874) — takes the B70's long-context prefill from ~780 to 1871 t/s at 32K, +26-29% at 4-8K, and fixes a multi-turn corruption bug (#25880). Measured A/B on Qwen3.6-35B MoE and dense 27B, then promoted to production.
Issue
Long-context prefill was 2.4× slower than it should have been, and b10222 carried a multi-turn corruption bug (SDPA scale use-after-return, #25880). Two upstream PRs that fix exactly this — #25874 (quantized-KV XMX FA) and #25880 — shipped after our pinned release, so the fixes were not in the production build.
Solution
Rebuilt llama.cpp SYCL from upstream master (commit 071327508, b10255+) into build-sycl-0804 with the same config (-O3 -DNDEBUG, DNN=ON, F16=ON, GRAPH=ON, oneAPI 2026.0). Ran a same-day A/B against b10222 on both MoE 35B and dense 27B, then promoted the strictly-better build to production via a single config value (profiles.json runtime.binary), backed up and health-verified.
local-aillama.cppsyclintel-arc
Everything you need to run production LLM inference on Intel Arc Pro B70 with llama.cpp SYCL — the build, runtime flags, all 5 model configs with measured VRAM boundaries, power tiers, KV cache KL-divergence analysis, and the b9853→b10222→master 0804 improvement data. 72.6 t/s MoE decode, 2128 t/s prefill, 512K context. Plus the dense 27B vLLM XPU track: 69.3 t/s MTP4 decode via GPTQ-INT4.
Issue
Running LLMs on Intel Arc requires SYCL-specific knowledge that doesn't exist in one place: which cmake flags, which env vars, which KV cache config, which power cap, which context length per model. Getting any of these wrong means either crashes, bad quality, or leaving performance on the table.
Solution
Fact-checked every claim against llama.cpp PRs and external benchmarks, ran a full boundary sweep measuring VRAM at every quant/context combo, A/B tested two SYCL builds, and consolidated everything into one recipe with the exact commands and measured numbers.
local-aillama.cppsyclintel-arc
Every config that works for running Qwen3.6 27B dense on Intel Arc Pro B70 with llama.cpp SYCL — Q4/Q5/Q6 quant comparison, MTP-4 speculative decoding (+35-50%), q8_0 K + q4_1 V KV cache with measured KL-divergence, and the exact VRAM context ceiling for each quant. Validated on build b10222 with llama-bench.
Issue
The 27B dense model is VRAM-hungry (16-21 GB weights + 7-14 GB KV at high context), making the quant/context/KV trade-off non-obvious. The previous fleet used q5_0-q4_1 KV cache, but KL-divergence analysis showed q8_0 K + q4_1 V is near-lossless. The context ceiling per quant was unknown.
Solution
Tested all three quants (Q4_K_M, Q5_K_M, Q6_K) at every context length (128K-512K) with q8_0 K + q4_1 V KV cache. Measured VRAM boundaries, decode speed, and prefill with llama-bench. MTP-4 speculative decoding adds +35-50% decode. The KV insight: dense models use 3.8x more KV cache than MoE — so a 16 GB dense model can't pass 256K while a 25 GB MoE reaches 512K.
local-aillama.cppsyclintel-arc
Running Poolside's 118B Laguna S 2.1 (256 experts, 8B active) on one Intel Arc Pro B70 32GB. 3.2x speedup from partial expert offload instead of dumping all experts to CPU. DFlash spec decode tested and dismissed.
Issue
Putting all experts on CPU gave 4.8 t/s and left 24 GB of VRAM idle. DFlash speculative decoding tested at 5.0 t/s, within noise of baseline.
Solution
Keep experts for layers 0-39 on GPU, only send layers 40-47 to CPU via -ot regex. 15.3 t/s. One flag change, no dependencies.
intel-arcb70llama.cppsycl
Real single-stream timings from a 2026-07-16 B70 suite: long-prompt prefill near 1.7k t/s on MoE, dense Gemma 31B +51% decode with MTP-4, and Grok Build CLI with tools enabled.
Issue
Short-prompt prefill looked terrible, dense Gemma needed a path that was not stuck at ~16 t/s, and agent CLI numbers were being confused with engine throughput.
Solution
Re-measure with engine timings at long prompt sizes, replace Gemma 26B with Gemma 4 31B + Unsloth MTP draft + vision mmproj, keep MoE profiles for speed, and run Grok Build with tools on.
intel-arcb70llama.cppsycl
Switching from symmetric q8_0 to asymmetric q5_0-q4_1 KV cache quantization freed 6.2 GB of VRAM per 128K context, pushed context ceilings to 256K on a 35B model, and was 3.3% faster in engine decode rate. Hardware-verified on Intel Arc Pro B70 32GB.
Issue
Standard q8_0 KV cache quantization used a 0.531 VRAM multiplier, capping a 35B Q5 model at 128K context on 32GB. The question was whether asymmetric K/V quantization (q5_0 for K, q4_1 for V) could unlock higher context lengths without hitting the quality cliff or degrading throughput.
Solution
Ran a 5-test hardware-verified benchmark suite on llama.cpp b9851 comparing q8_0-q8_0 against q5_0-q4_1 across control baseline, target comparison, flagship configs, and dense model validation. Calculated per-model VRAM budgets using measured multipliers from the Anbeeld 2026 KV cache benchmark methodology.
local-aillama.cppkv-cachequantization
Corrected power scaling data for Qwen 27B MTP-4 speculative decoding after discovering single-prompt caching was inflating baselines by 4-5%. True MTP-4 gain at 180W is +35%, not +41%. Includes vision benchmark results after ffmpeg dependency fix.
Issue
The initial power sweep data was inflated. Single-prompt prefix caching was active during testing, which inflated the baseline measurements by approximately 4-5%. This made the speculative decoding gains appear larger than they actually were.
Solution
Rewrote the benchmark script to enforce warmup discards, isolate engine decode rate from wall-clock time, and maintain strict thermal cooldowns between test rounds. Re-ran the entire power sweep with the corrected methodology.
local-aillama.cppmtpspeculative-decoding
A reproducible case study for running Qwen3.6-35B-A3B on Intel Arc Pro B70 with llama.cpp SYCL on Ubuntu 26.04, including the exact build, runtime flags, benchmark data, and the persistent SYCL cache issue that caused model-load crashes.
Issue
The obvious checks all passed: the GPU was visible through Level Zero, ReBAR exposed the full 32GB BAR, the model fit in VRAM, and Vulkan could load it. SYCL still failed during model load, first with xe bcs engine resets and then with SIGSEGV crashes even when GPU offload was disabled.
Solution
I rebuilt llama.cpp from current master with Level Zero development headers installed, disabled Intel SYCL persistent kernel cache, pinned the Level Zero device explicitly, and reduced the environment to the smallest set of variables required for stable SYCL inference.
local-aillama.cppsyclintel-arc
Full benchmark data on running MoE and dense LLMs on AMD consumer hardware — quantization comparison, power cap analysis, KV cache tuning, and context limits on 16GB VRAM.
Issue
Consumer GPUs have hard VRAM ceilings. Running 23-35B parameter models on 16GB requires aggressive quantization, KV cache compression, and precise build flags. The noise-to-signal ratio in online benchmarking is high — most people test on NVIDIA, not AMD RDNA3, and few test MoE architectures with context windows above 32K.
Solution
Systematically benchmarked 8+ models across 5 quantization levels, swept GPU power caps from 30W to 190W, tested 3 KV cache configurations, and pushed context limits to 256K. Documented the exact llama.cpp build flags and runtime parameters that make 128K inference on 16GB VRAM stable and fast.
local-aillama.cpprocmamd
A practical guide to splitting an oversized Git PR into clean, topic-focused branches using path-based checkout from a fresh branch off main.
Issue
Mixed branches make PRs unreviewable, increase blast radius, and risk dragging unrelated changes into production. When one branch contains role code, host variables, certificate files, and inventory updates together, reviewers cannot isolate what changed or why.
Solution
Split the oversized branch into multiple clean, topic-focused branches by checking out only the relevant paths from the mixed branch into new branches created fresh off main.
gitdevopsansibleworkflow
A master reference merging daily Linux operations, Ansible Vault secrets, Python environments, Molecule testing, networking diagnostics, and Git recovery commands into a single, massive cheatsheet.
Issue
Scattered knowledge means slower response times during critical operations. Having Linux commands on one page and Ansible/Python commands on another breaks the operational flow.
Solution
Compiled every sanitized, production-tested command snippet from my daily workflow into a single, massive reference guide with a coordinated SVG poster set.
linuxansiblepythongit
A case study on resolving application vs network conflicts by migrating from legacy OS-level IP aliasing to a robust reverse proxy architecture.
linuxarchitectureproxynetworking
A sanitized collection of Linux, storage, permissions, Git, cron, and Vim commands I keep close during day-to-day operations.
Issue
The knowledge existed, but it was fragmented across storage work, account management, package checks, Git recovery, and automation workflows. That fragmentation increases the chance of typos and slows down repeat work.
Solution
Consolidated the most reused Linux and admin commands into a snippets-first cheatsheet, grouped them by task, added flag guidance, and replaced every real identifier with placeholders.
linuxrhelbashgit
How to use DNS record management and staged certificate deployment to migrate critical services without service interruption.
Issue
Direct IP replacement would cause service disruption. Applications had hardcoded references to old hostnames. Certificates were tied to specific DNS names. Testing needed to happen in parallel with production operation.
Solution
Implemented a two-phase DNS migration strategy using temporary test records, multi-SAN certificates, and coordinated DNS switchover during a planned maintenance window.
dnsmigrationcertificateslinux
A complete guide to integrating Linux with Active Directory: mapping AD groups to local permissions, deploying Kerberos SSO, and troubleshooting PAM issues.
Issue
AD integration was fragmented across multiple playbooks with no unified approach. Users couldn't 'su' to service accounts, SSO setup was manual and error-prone, and access control required manual sudoers edits on each server.
Solution
Implemented a unified AD integration strategy: AD group mapping for sudo access, automated Kerberos keytab deployment via Ansible, and standardized PAM configuration across all servers.
linuxactive-directorykerberossssd
A complete guide to Logical Volume Manager operations—expanding partitions online, shrinking safely, and migrating directories with minimal downtime.
Issue
Storage operations were handled inconsistently across the team. Some admins would reboot servers for partition changes, others would attempt risky online operations without proper checkpoints, and migrations often resulted in extended downtime windows.
Solution
Documented a standardized LVM playbook covering the three core operations—expansion, shrinking, and migration—with clear pre-flight checks, execution steps, and rollback procedures.
lvmstoragesysadminrhel
How to provision, mount, and troubleshoot NFS exports across enterprise Linux servers using Ansible, LVM, and proper network segmentation.
Issue
NFS configuration was inconsistent across servers. Some used hostnames, others used IPs. Network routing issues caused connections over slow backup networks instead of high-bandwidth production networks. Permission errors blocked user access.
Solution
Implemented automated NFS management using Ansible roles for export configuration, client mounting with proper network selection, and troubleshooting runbooks for common failure scenarios.
nfsstorageansiblelinux
A practical guide to the Linux server provisioning workflow—from creating AD groups and technical users to Ansible role deployment and application-specific configurations.
Issue
Server provisioning was inconsistent across team members. Some skipped steps, documentation was scattered across wikis and emails, and handoffs to application teams were incomplete—missing access groups, wrong technical user configurations, or incomplete application dependencies.
Solution
Developed a standardized provisioning checklist and Ansible playbook structure that covers the complete lifecycle from VM deployment to application-ready state.
provisioningansiblerhelactive-directory
A concrete security module set for an edge AI backend: AES-256-GCM at rest, adaptive rate limiting, input validation, alerting, and automated scanning.
Issue
Without explicit controls, an AI API is vulnerable to abuse (burst traffic), unsafe inputs (command/path traversal), leaked secrets, and silent security regressions from dependencies.
Solution
Implemented five security modules: encryption at rest, enhanced rate limiting, advanced input validation, security monitoring + alerts, and vulnerability scanning with report generation.
securitynodejstypescriptrate-limiting
Understanding stretched Layer 2 networks across data centers, leaf-spine fabric design, and the trade-offs for enterprise applications.
Issue
Lack of understanding about stretched networks, leaf-spine trade-offs, and how application traffic patterns would be affected.
Solution
Documented the stretched network architecture, analyzed application traffic flows, and provided clear guidance on which applications were suitable for stretched L2 vs. Layer 3 approaches.
networkingleaf-spinedisaster-recoveryarchitecture
Patterns for automating silent software installations on Linux, handling response files, pre-requisite checks, and idempotent deployments.
Issue
Manual software installations were time-consuming, inconsistent across servers, and couldn't be reproduced reliably for disaster recovery.
Solution
Developed Ansible patterns for silent installations with templated response files, pre-requisite validation, and idempotent deployment checks.
ansiblesilent-installenterprise-softwareautomation
How to package Ansible dependencies into a portable, containerized Execution Environment (EE) for consistent automation across runners.
Issue
Ansible playbooks that worked on the control node failed on execution environments with missing dependencies, and reproducing issues was difficult without consistent environments.
Solution
Built custom Execution Environments using ansible-builder, packaging all Python dependencies, Ansible collections, and system packages into versioned container images.
ansiblecontainersdevopsautomation
Setting up PostgreSQL WAL archiving for point-in-time recovery, with SELinux context handling for archive directories.
Issue
No WAL archiving configured, SELinux contexts incorrect for archive directories, and point-in-time recovery was impossible.
Solution
Configured PostgreSQL WAL archiving with proper SELinux file contexts, tested restore procedures, and documented the end-to-end recovery process.
postgresqlbackupselinuxdatabase
How to deploy and configure Apache as a reverse proxy with Ansible, including SSL termination, load balancing, and health checks.
Issue
No consistent reverse proxy pattern, manual SSL certificate management, and inconsistent load balancer configurations across environments.
Solution
Developed an Ansible role for Apache reverse proxy with automated SSL deployment, health check endpoints, and standardized load balancer configurations.
ansibleapachereverse-proxyssl
How to create standardized golden images for VMware using Packer with StackGuardian integration for automated image pipelines.
Issue
No standardized golden images, manual image building was error-prone, and configuration drift between images caused deployment failures.
Solution
Implemented Packer with StackGuardian for automated golden image pipelines, creating standardized RHEL images with consistent configurations.
packervmwaregolden-imageautomation
How to structure Ansible patching playbooks into controlled waves with health checks, rollback triggers, and clear ownership boundaries.
Issue
Big-bang patching caused widespread outages with no rollback strategy, and identifying affected systems took hours during incidents.
Solution
Implemented wave-based patching with health gates between waves, automatic rollback triggers, and per-wave ownership documentation.
ansiblepatchingrhellifecycle
How to set up automated testing for Ansible roles using Molecule with Docker drivers, ensuring playbooks work before production deployment.
Issue
No automated testing for Ansible roles, production deployments were the first test, and role regressions were discovered only after incidents.
Solution
Implemented Molecule with Docker for local role testing, integrated into CI pipeline to catch issues before merge.
ansiblemoleculedockertesting
A practical pattern for managing local users, groups, and sudo access across Linux servers using Ansible with host-specific variables.
Issue
No centralized user management for local accounts, UID/GID inconsistencies breaking applications, and sudo access scattered across individual sudoers files.
Solution
Implemented Ansible-based user management with host_vars for server-specific accounts, standardized UID/GID ranges, and templated sudoers configurations.
ansiblelinuxuser-managementsudo
A practical bash script to audit user permissions, sudo access, and group memberships across Linux servers for compliance reporting.
Issue
No automated way to gather user permission data, manual auditing was error-prone and time-consuming, and compliance reports were always delayed.
Solution
Developed a bash script that collects user accounts, sudo access, and group memberships, outputting a standardized report that could be consolidated across all servers.
bashlinuxauditcompliance
How to use the tracer utility to identify which services need restart after package updates, and plan reboots strategically across server tiers.
Issue
No visibility into which services had pending restarts, leading to either unnecessary reboots or missed restarts that caused instability.
Solution
Implemented tracer integration to identify pending restarts, combined with a tiered reboot strategy based on application criticality.
rheltracerpatchinglifecycle
A practical cheatsheet covering the most essential commands for managing RHEL systems on a daily basis: systemd, storage, networking, and user management.
Issue
No single source of truth for common RHEL administration commands, leading to inconsistent practices and repeated onboarding questions.
Solution
Created a living cheatsheet covering systemd, LVM, networking, user management, and troubleshooting - the commands used daily in our environment.
LinuxRHELSystemAdminCLI
Best practices for organizing your Ansible inventory, group_vars, and host_vars to cleanly separate development and production environments.
Issue
No clear separation between dev and prod environments, inconsistent variable hierarchy, and accidental cross-environment changes were becoming common.
Solution
Implemented a standardized repository structure with separate inventory directories, clear group_vars/host_vars hierarchy, and environment-specific variable overrides.
ansibleiacdevopsarchitecture