gavin/transformers
1
0
Fork 0
Code Issues Pull Requests Actions 9 Packages Projects Releases Wiki Activity
Files
main
transformers/docs/source/en/main_classes/quantization.md
陈赣 06f1fd69a6
Some checks failed
Self-hosted runner (nightly-past-ci-caller) / Get number (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.11 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.10 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.9 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.8 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.7 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.6 (push) Has been cancelled
Details
Self-hosted runner (nightly-past-ci-caller) / TensorFlow 2.5 (push) Has been cancelled
Details
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Has been cancelled
Details
Build documentation / build (push) Has been cancelled
Details
Build documentation / build_other_lang (push) Has been cancelled
Details
CodeQL Security Analysis / CodeQL Analysis (push) Has been cancelled
Details
New model PR merged notification / Notify new model (push) Has been cancelled
Details
PR CI / pr-ci (push) Has been cancelled
Details
Slow tests on important models (on Push - A10) / Get all modified files (push) Has been cancelled
Details
Secret Leaks / trufflehog (push) Has been cancelled
Details
Update Transformers metadata / build_and_package (push) Has been cancelled
Details
Slow tests on important models (on Push - A10) / Model CI (push) Has been cancelled
Details
Check Tiny Models / Check tiny models (push) Has been cancelled
Details
Self-hosted runner (Intel Gaudi3 scheduled CI caller) / Model CI (push) Has been cancelled
Details
Self-hosted runner (Intel Gaudi3 scheduled CI caller) / Pipeline CI (push) Has been cancelled
Details
Self-hosted runner (Intel Gaudi3 scheduled CI caller) / Example CI (push) Has been cancelled
Details
Self-hosted runner (Intel Gaudi3 scheduled CI caller) / DeepSpeed CI (push) Has been cancelled
Details
Self-hosted runner (Intel Gaudi3 scheduled CI caller) / Trainer/FSDP CI (push) Has been cancelled
Details
Nvidia CI - Flash Attn / Setup (push) Has been cancelled
Details
Nvidia CI - Flash Attn / Model CI (push) Has been cancelled
Details
Nvidia CI / Setup (push) Has been cancelled
Details
Nvidia CI / Model CI (push) Has been cancelled
Details
Nvidia CI / Torch pipeline CI (push) Has been cancelled
Details
Nvidia CI / Example CI (push) Has been cancelled
Details
Nvidia CI / Trainer/FSDP CI (push) Has been cancelled
Details
Nvidia CI / DeepSpeed CI (push) Has been cancelled
Details
Nvidia CI / Quantization CI (push) Has been cancelled
Details
Nvidia CI / Kernels CI (push) Has been cancelled
Details
Doctests / Setup (push) Has been cancelled
Details
Doctests / Call doctest jobs (push) Has been cancelled
Details
Doctests / Send results to webhook (push) Has been cancelled
Details
Extras Smoke Test / Get supported Python versions (push) Has been cancelled
Details
Extras Smoke Test / Test extras on Python ${{ matrix.python-version }} (push) Has been cancelled
Details
Extras Smoke Test / Check Slack token availability (push) Has been cancelled
Details
Extras Smoke Test / Notify failures to Slack (push) Has been cancelled
Details
Self-hosted runner (AMD scheduled CI caller) / Trigger Scheduled AMD CI (push) Has been cancelled
Details
Stale Bot / Close Stale Issues (push) Has been cancelled
Details
first commit
2026-06-05 16:53:03 +08:00

2.4 KiB
Executable File
Raw Permalink Blame History

Quantization

Quantization techniques reduce memory and computational costs by representing weights and activations with lower-precision data types like 8-bit integers (int8). This enables loading larger models you normally wouldn't be able to fit into memory, and speeding up inference. Transformers supports the AWQ and GPTQ quantization algorithms and it supports 8-bit and 4-bit quantization with bitsandbytes.

Quantization techniques that aren't supported in Transformers can be added with the [HfQuantizer] class.

Learn how to quantize models in the Quantization guide.

QuantoConfig

autodoc QuantoConfig

AqlmConfig

autodoc AqlmConfig

VptqConfig

autodoc VptqConfig

AwqConfig

autodoc AwqConfig

EetqConfig

autodoc EetqConfig

GPTQConfig

autodoc GPTQConfig

BitsAndBytesConfig

autodoc BitsAndBytesConfig

HfQuantizer

autodoc quantizers.base.HfQuantizer

HiggsConfig

autodoc HiggsConfig

HqqConfig

autodoc HqqConfig

MetalConfig

autodoc MetalConfig

Mxfp4Config

autodoc Mxfp4Config

FbgemmFp8Config

autodoc FbgemmFp8Config

CompressedTensorsConfig

autodoc CompressedTensorsConfig

TorchAoConfig

autodoc TorchAoConfig

BitNetQuantConfig

autodoc BitNetQuantConfig

SpQRConfig

autodoc SpQRConfig

FineGrainedFP8Config

autodoc FineGrainedFP8Config

QuarkConfig

autodoc QuarkConfig

FourOverSixConfig

autodoc FourOverSixConfig

FPQuantConfig

autodoc FPQuantConfig

AutoRoundConfig

autodoc AutoRoundConfig

SinqConfig

autodoc SinqConfig