Intel Xeon
intel-xeon-5381b7a6·5 events·first seen 28d agoAliases: Intel Xeon
Co-occurring entities
More like this (12)
Recent events (5)
Q8-Chat: Efficient Generative AI on Intel Xeon via INT8 Quantization
Hugging Face and Intel demonstrate running quantized large language models (INT8/Q8) on Intel Xeon CPUs, branded as Q8-Chat. The post covers inference performance of quantized models on CPU hardware without requiring GPUs. This is relevant to inference economics and enterprise deployment, particularly for organizations without GPU infrastructure.
Google Cloud C4 Brings a 70% TCO Improvement on GPT OSS with Intel and Hugging Face
A collaboration between Google Cloud, Intel, and Hugging Face demonstrates a 70% total cost of ownership (TCO) reduction when running open-source GPT-class models on Google Cloud's C4 instances powered by Intel Xeon processors. The post details inference economics for deploying open-weight LLMs on CPU-based cloud infrastructure rather than GPU instances. This represents a notable data point in the inference cost optimization space, particularly for organizations seeking lower-cost alternatives to GPU-based deployment.
Building Cost-Efficient Enterprise RAG Applications with Intel Gaudi 2 and Intel Xeon
This Hugging Face blog post details how to build retrieval-augmented generation (RAG) pipelines for enterprise use cases using Intel Gaudi 2 accelerators and Intel Xeon CPUs. It covers the architecture and cost-efficiency tradeoffs of deploying RAG on Intel hardware as an alternative to GPU-based infrastructure. The post is positioned as a practical guide for organizations seeking lower-cost inference deployments.
Blazing Fast SetFit Inference with Optimum Intel on Xeon
Hugging Face demonstrates accelerated inference for SetFit few-shot text classification models using Optimum Intel on Intel Xeon CPUs. The post covers optimization techniques such as quantization and ONNX export to improve throughput and latency for CPU-based deployment. This is relevant to practitioners deploying lightweight NLP models in cost-sensitive or edge environments without GPU hardware.
Accelerate StarCoder with Optimum Intel on Xeon: Q8/Q4 and Speculative Decoding
Hugging Face and Intel demonstrate quantization (INT8/INT4) and speculative decoding techniques applied to StarCoder on Intel Xeon CPUs using the Optimum Intel library. The post covers practical inference acceleration workflows targeting CPU deployment of code generation models. This represents a concrete inference-economics use case for open-weight code models on commodity server hardware.