SergiioB/intel-arc-pro-b70-inference-cookbook

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.

Everything you need to know about LLM inference

Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

~1300 line, minimal, extensible LLM inference engine built from scratch.

Experimental llama.cpp fork for inference research and development

100 days of LLM inference engineering — daily posts, experiments, and visualizations

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.