transformers/docs/source/en/pr_checks.md

Pull request checks

When you open a pull request, the Hugging Face CI runs several checks that must pass before your PR can be merged.

• Fixing the CI lists the commands to run locally to pass the checks.
• The sections after that describe what each check validates, such as code quality and repository consistency.
• Tests describe which tests are run, test categories, and slow tests.

Fixing the CI

In most cases, make style is enough to clear the code quality check, which is the most common failure.

make style

For failures in repository consistency, copies, or auto-generated files, run make fix-repo. It fixes style, copies, docstrings, and auto-generated files in one pass.

make fix-repo

For a larger change, the three-command sequence below covers every check. It is heavier, but catches everything before you push.

make fix-repo   # auto-fix everything that can be auto-fixed
make typing     # check types and model structure, fix any errors manually
make check-repo # verify all checks pass, fix anything that remains

make typing catches type errors and model structure violations that you fix manually. make check-repo does a final read-only pass so you can confirm everything is ready.

Note

The CI is occasionally flaky. If a check fails on something unrelated to your change, ping a maintainer to re-run it.

Code quality

The code quality check covers formatting, imports, type checking, and model structure rules. It corresponds to make fix-repo and make typing.

make style (included in make fix-repo) auto-fixes Ruff linting and formatting, __init__.py import sort order, and auto-mapping consistency.

make typing performs type checking with ty and validates TRansFormers (TRF) rules, which cover config class naming conventions and forward() signatures. Type errors and TRF violations report a specific rule number and must be fixed manually. The rules live in the mlinter repository. Run python -m utils.mlinter --list-rules to see every TRF rule, or python -m utils.mlinter --rule TRFXXX to view the full documentation for a specific rule.

If a TRF rule needs an exception, choose one of these options (see Suppressing violations for more details).

• Add your model name to the allowlist_models list for the relevant rule in utils/mlinter/rules.toml. Use this when the whole model file needs an exception.
• Add # trf-ignore: TRFXXX on the same line as the flagged construct, or on the line immediately above it. Use this when only one flagged construct needs an exception.

Repository consistency

The repository consistency check is similar to make check-repo, except it stops on the first failure. It keeps the repository internally consistent across the categories below: public objects stay importable, copied code stays in sync with its source, and auto-generated files (dummies, doctests, metadata) reflect the current state of the code. For new models, it also verifies every new model class is registered in the auto-mappings.

Category	What it validates	Auto-fixed?
Init files	Every new public object must appear in both _import_structure (lazy loading) and the if TYPE_CHECKING block (type checker imports) in __init__.py	Manual
Copies and modular	# Copied from blocks match their source and modular-generated files are up to date	make fix-repo
Docstrings and docs	Argument docstrings match function signatures and documentation table of contents	make fix-repo
Auto-generated files	Dummies, pipeline typing, doctest list, metadata, dependency table	make fix-repo
Config validation	Config classes have valid checkpoints in docstrings and config attributes match modeling file	Manual

Tests

CI runs a targeted subset of tests based on what your PR changes. The CI runs tests in a slightly randomized order with pytest-random-order to catch coupled tests. The run prints the random seed at the start so you can replay the same order with --random-order-seed=<seed>.

If a test passes locally on GPU but fails in CI, set TRANSFORMERS_TEST_DEVICE="cpu" to check whether you can reproduce the failure on CPU.

TRANSFORMERS_TEST_DEVICE="cpu" pytest tests/models/my_model/ -v

The sections below explain how test selection works, which jobs run, and how to handle slow tests.

Test selection

CI doesn't run the full test suite on every PR. CI skips tests decorated with @slow on every PR, and a maintainer triggers them on GPU once the PR is under review.

utils/tests_fetcher.py traces import dependencies from your changed files to identify affected tests, and only runs those. It also catches regressions in other models when you touch shared utilities. The fetcher prints which files changed and which tests are impacted, then writes the list to tests_torch_test_list.txt.

Use the fetcher to replicate exactly what CI runs.

python utils/tests_fetcher.py
python -m pytest -n 8 --dist=loadfile -rA -s $(cat test_preparation/tests_torch_test_list.txt)

Tip

Changing core files like modeling_utils.py or generation/utils.py triggers all model tests, not just the affected subset.

You can also run every test for your model unconditionally. A direct run is a faster local sanity check, but it won't catch regressions in other models caused by shared code you touched.

pytest tests/models/my_model/ -v

Test job categories

Tests are split across parallel CI jobs, and each job picks up files by path pattern. The relevant jobs for a model PR are:

• tests_torch: modeling tests (tests/models/*/test_modeling_*.py)
• tests_tokenization: tokenizer tests (tests/models/*/test_tokenization_*.py)
• tests_processors: processor and feature extractor tests (tests/models/*/test_(processing|image_processing|feature_extractor)_*.py)
• tests_generate: generation tests
• pipelines_torch: pipeline tests
• tests_training_ci: training loop tests
• tests_tensor_parallel_ci: tensor parallel tests

Slow tests

Regular CI runs skip tests decorated with @slow. They download real checkpoints or need significant compute, so they run on GPU instances, and maintainers trigger them once your PR is under review.

Slow tests run on an NVIDIA A10, and numerical results can vary slightly between the CI hardware and your local machine. Maintainers usually adjusts those values on tests if needed when adding a new model.

Run slow tests locally with the command below.

RUN_SLOW=1 python -m pytest tests/models/my_model/ -v

Documentation build

A build_pr_documentation job builds and generates a preview of the documentation. A bot posts a preview link in your PR, and the check must pass before merging. Most failures are a missing entry in the toctree. To build the documentation locally, see the README.md in the docs folder.

# Copied from syntax

Warning

For new models, always prefer the modular workflow (modular_*.py) over # Copied from. Avoid # Copied from whenever possible.

The # Copied from mechanism keeps copied code in sync with its source. When make fix-repo runs, it checks every # Copied from block and updates it to match the original, so edits inside a # Copied from block are overwritten. Edit the source instead and let make fix-repo propagate the change.

The basic forms of # Copied from include the following.

# Copied from transformers.models.bert.modeling_bert.BertSelfOutput

# Copied from transformers.models.bert.modeling_bert.BertAttention with Bert->Roberta

# Copied from transformers.models.bert.modeling_bert.BertForSequenceClassification with Bert->MobileBert all-casing

The with model->newModel syntax applies string replacements after copying. Separate multiple replacements with commas, which are applied from left to right. The all-casing option replaces every casing variant at once (Bert, bert, BERT become MobileBert, mobilebert, MOBILEBERT).