first commit

docs/source/en/community_integrations/axolotl.md

@@ -0,0 +1,60 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Axolotl
+
+[Axolotl](https://docs.axolotl.ai/) is a fine-tuning and post-training framework for large language models. It supports adapter-based tuning, ND-parallel distributed training, GRPO, and QAT. Through [TRL](./trl), Axolotl also handles preference learning, reinforcement learning, and reward modeling workflows.
+
+Define your training run in a YAML config file.
+
+```yaml
+base_model: NousResearch/Nous-Hermes-llama-1b-v1
+model_type: AutoModelForCausalLM
+tokenizer_type: AutoTokenizer
+
+datasets:
+  - path: tatsu-lab/alpaca
+    type: alpaca
+
+output_dir: ./outputs
+sequence_len: 512
+micro_batch_size: 1
+gradient_accumulation_steps: 1
+num_epochs: 1
+learning_rate: 2.0e-5
+```
+
+Launch training with the [train](https://docs.axolotl.ai/docs/cli.html#train) command.
+
+```bash
+axolotl train my_config.yml
+```
+
+## Transformers integration
+
+Axolotl's [ModelLoader](https://docs.axolotl.ai/docs/api/loaders.model.html#axolotl.loaders.model.ModelLoader) wraps the Transformers load flow.
+
+- The model config builds from [`AutoConfig.from_pretrained`]. Preload setup configures the [device map](https://huggingface.co/docs/accelerate/concept_guides/big_model_inference#designing-a-device-map), [quantization config](../main_classes/quantization), and [attention backend](../attention_interface).
+
+- `ModelLoader` automatically selects the appropriate [`AutoModel`] class ([`AutoModelForCausalLM`], [`AutoModelForImageTextToText`], [`AutoModelForSequenceClassification`]) or a model-specific class from the multimodal mapping. Weights load with the selected loader's `from_pretrained`. When `reinit_weights` is set, Axolotl uses `from_config` for random initialization.
+
+- Axolotl uses Transformers, [PEFT](https://huggingface.co/docs/peft/index), and [bitsandbytes](https://huggingface.co/docs/bitsandbytes/index) to apply adapters after model initialization when PEFT-based techniques such as LoRA and QLoRA are enabled. A patch manager applies additional optimizations before and after model loading.
+
+- [AxolotlTrainer](https://docs.axolotl.ai/docs/api/core.trainers.base.html#axolotl.core.trainers.base.AxolotlTrainer) extends [`Trainer`], adding Axolotl mixins while using the [`Trainer`] training loop and APIs.
+
+## Resources
+
+- [Axolotl](https://docs.axolotl.ai/) docs

docs/source/en/community_integrations/candle.md

@@ -0,0 +1,43 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Candle
+
+[Candle](https://github.com/huggingface/candle) is a machine learning framework providing native Rust implementations of Transformers models. It natively supports [safetensors](https://huggingface.co/docs/safetensors/en/index) to load Transformers models directly.
+
+```rust
+/// load model config
+let config: Config = 
+    serde_json::from_reader(std::fs::File::open(config_filename)?)?;
+
+/// load safetensors and memory-maps them
+let vb = unsafe {
+    VarBuilder::from_mmaped_safetensors(&filenames, dtype, &device)?
+};
+
+/// materialize tensors from VarBuilder into model class
+let model = Model::new(args.use_flash_attn, &config, vb)?;
+```
+
+## Transformers integration
+
+1. The [hf-hub](https://github.com/huggingface/hf-hub) crate checks your local [Hugging Face cache](../installation#cache-directory) for a model. If it isn't there, it downloads model weights and configs from the Hub.
+2. [VarBuilder](https://github.com/huggingface/candle/blob/f526033db7ea880c7189628a2dc00e3e2008a9e7/candle-nn/src/var_builder.rs#L38) lazily loads the safetensor files. It maps state-dict key names to Rust structs representing model layers. This mirrors how Transformers organizes its weights.
+3. Candle parses `config.json` to extract model metadata and instantiates the matching Rust model class with weights from `VarBuilder`.
+
+## Resources
+
+- [Candle](https://github.com/huggingface/candle) documentation

docs/source/en/community_integrations/executorch.md

@@ -0,0 +1,92 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# ExecuTorch
+
+[ExecuTorch](https://docs.pytorch.org/executorch/stable/index.html) is a lightweight runtime for model inference on edge devices. It exports a PyTorch model into a portable, ahead-of-time format. A small C++ runtime plans memory and dispatches operations to hardware-specific backends. Execution and memory behavior is known before the model runs on device, so inference overhead is low.
+
+Export a Transformers model with the [optimum-executorch](https://huggingface.co/docs/optimum-executorch/en/index) library.
+
+<hfoptions id="export">
+<hfoption id="CLI">
+
+```bash
+optimum-cli export executorch \
+    --model "HuggingFaceTB/SmolLM2-135M-Instruct" \
+    --task "text-generation" \
+    --recipe "xnnpack" \
+    --output_dir="./smollm2_exported"
+```
+
+</hfoption>
+<hfoption id="Python">
+
+```py
+from transformers import AutoTokenizer
+from optimum.executorch import ExecuTorchModelForCausalLM
+
+model = ExecuTorchModelForCausalLM.from_pretrained(
+    "HuggingFaceTB/SmolLM2-135M-Instruct",
+    recipe="xnnpack",
+)
+model.save_pretrained("./smollm2_exported")
+tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-135M-Instruct")
+```
+
+</hfoption>
+</hfoptions>
+
+## Transformers integration
+
+The export process uses several Transformers components.
+
+1. [`~PreTrainedModel.from_pretrained`] loads the model weights in safetensors format.
+2. Optimum applies graph optimizations and runs [torch.export](https://docs.pytorch.org/docs/stable/export.html) to create a `model.pte` file targeting your hardware backend.
+3. [`AutoTokenizer`] or [`AutoProcessor`] loads the tokenizer or processor files and runs during inference.
+4. At runtime, a C++ runner class executes the `.pte` file on the ExecuTorch runtime.
+
+```c++
+#include <executorch/extension/llm/runner/text_llm_runner.h>
+
+using namespace executorch::extension::llm;
+
+int main() {
+  // Load tokenizer and create runner
+  auto tokenizer = load_tokenizer("path/to/tokenizer.json", nullptr, std::nullopt, 0, 0);
+  auto runner = create_text_llm_runner("path/to/model.pte", std::move(tokenizer));
+
+  // Load the model
+  runner->load();
+
+  // Configure generation
+  GenerationConfig config;
+  config.max_new_tokens = 100;
+  config.temperature = 0.8f;
+
+  // Generate text with streaming output
+  runner->generate("The capital of France is", config,
+    [](const std::string& token) { std::cout << token << std::flush; },
+    nullptr);
+
+  return 0;
+}
+```
+
+## Resources
+
+- [ExecuTorch](https://docs.pytorch.org/executorch/stable/index.html) docs
+- [torch.export](https://docs.pytorch.org/docs/stable/export.html) docs
+- [Exporting to production](../serialization#executorch) guide

docs/source/en/community_integrations/llama_cpp.md

@@ -0,0 +1,61 @@
+<!--Copyright 2025 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# llama.cpp
+
+[llama.cpp](https://github.com/ggml-org/llama.cpp) is a C/C++ inference engine for deploying large language models locally. It's lightweight and doesn't require Python, CUDA, or other heavy server infrastructure. llama.cpp uses the [GGUF](https://huggingface.co/blog/ngxson/common-ai-model-formats#gguf) file format. GGUF supports quantized model weights and memory-mapping to reduce memory bandwidth on your device.
+
+> [!TIP]
+> Browse the [Hub](https://huggingface.co/models?apps=llama.cpp&sort=trending) for models already available in GGUF format.
+
+Convert any Transformers model to GGUF format with the [convert_hf_to_gguf.py](https://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py) script.
+
+```bash
+python3 convert_hf_to_gguf.py ./models/openai/gpt-oss-20b \
+  --outfile gpt-oss-20b.gguf \
+```
+
+Deploy the model locally from the command line with [llama-cli](https://github.com/ggml-org/llama.cpp/tree/master#llama-cli) or start a web UI with [llama-server](https://github.com/ggml-org/llama.cpp/tree/master#llama-server). Add the `-hf` flag to indicate the model is from the Hub.
+
+<hfoptions id="deploy">
+<hfoption id="llama-cli">
+
+```bash
+llama-cli -hf ggml-org/gpt-oss-20b-GGUF
+```
+
+</hfoption>
+<hfoption id="llama-server">
+
+```bash
+llama-server -hf ggml-org/gpt-oss-20b-GGUF
+```
+
+</hfoption>
+</hfoptions>
+
+## Transformers integration
+
+1. [`AutoConfig.from_pretrained`] loads the model's `config.json` file to extract metadata.
+2. [`AutoTokenizer.from_pretrained`] extracts the vocabulary and tokenizer configuration.
+3. Based on the `architectures` field in the config, the script selects a converter class from its internal registry. The registry maps Transformers architecture names (like [`LlamaForCausalLM`]) to corresponding converter classes.
+4. The converter maps Transformers tensor names (for example, `model.layers.0.self_attn.q_proj.weight`) to GGUF tensor names, transforms tensors, and packages the vocabulary.
+5. The output is a single GGUF file containing the model weights, tokenizer, and metadata.
+
+## Resources
+
+- [llama.cpp](https://github.com/ggml-org/llama.cpp) documentation
+- [Introduction to ggml](https://huggingface.co/blog/introduction-to-ggml) blog post

docs/source/en/community_integrations/mlx.md

@@ -0,0 +1,54 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# MLX
+
+[MLX](https://ml-explore.github.io/mlx/build/html/index.html) is an array framework for machine learning on Apple silicon that also works with CUDA. On Apple silicon, arrays stay in shared memory to avoid data copies between CPU and GPU. Lazy computation enables graph manipulation and optimizations. Native [safetensors](https://huggingface.co/docs/safetensors/en/index) support means Transformers language models run directly on MLX.
+
+Install the [mlx-lm](https://github.com/ml-explore/mlx-lm) library.
+
+```bash
+pip install mlx-lm transformers
+```
+
+Load any Transformers language model from the Hub as long as the model architecture is [supported](https://huggingface.co/mlx-community/models). No weight conversion is required.
+
+```py
+from mlx_lm import load, generate
+
+model, tokenizer = load("openai/gpt-oss-20b")
+output = generate(
+    model,
+    tokenizer,
+    prompt="The capital of France is",
+    max_tokens=100,
+)
+print(output)
+```
+
+## Transformers integration
+
+- [mlx_lm.load](https://github.com/ml-explore/mlx-lm?tab=readme-ov-file#python-api) loads safetensor weights and returns a model and tokenizer.
+- MLX loads weight arrays keyed by tensor names and maps them into an MLX [nn.Module](https://ml-explore.github.io/mlx/build/html/python/nn/module.html#) parameter tree. This matches how Transformers checkpoints are organized.
+
+> [!TIP]
+> The MLX Transformers integration is bidirectional. Transformers can also load and run MLX weights from the Hub.
+
+## Resources
+
+- [MLX](https://ml-explore.github.io/mlx/build/html/index.html) documentation
+- [mlx-lm](https://github.com/ml-explore/mlx-lm) repository containing MLX LLM implementations
+- [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) community library with VLM implementations

docs/source/en/community_integrations/nanotron.md

@@ -0,0 +1,40 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Nanotron
+
+[Nanotron](https://github.com/huggingface/nanotron) is a distributed training framework with tensor, parallel, and data parallelism (3D parallelism). It is designed for large-scale training workloads across hundreds of GPUs.
+
+Convert any Transformers model to an optimized Nanotron transformer model implementation for pretraining with the [convert_hf_to_nanotron.py](https://github.com/huggingface/nanotron/blob/main/examples/llama/convert_hf_to_nanotron.py) script.
+
+```bash
+torchrun --nproc_per_node=1 examples/llama/convert_hf_to_nanotron.py \
+    --checkpoint_path=meta-llama/Llama-2-7b-hf \
+    --save_path=./llama-7b-nanotron
+```
+
+## Transformers integration
+
+1. Load a supported Transformers model, like [`Llama`], with the [`~LlamaForCausalLM.from_pretrained`] function. This reads the `config.json` file from the checkpoint directory and creates a [`LlamaConfig`].
+2. Nanotron maps [`LlamaConfig`] to it's own config format and creates a Nanotron model.
+3. Convert Transformers weights to Nanotron. A weight mapping guides how to map Nanotron parameter names to Transformers parameter names. This includes handling transformations such as fusing the QKV projections and the gate/up projections.
+
+Nanotron also relies on [`AutoTokenizer`] for turning text into token ids during preprocessing and generation.
+
+## Resources
+
+- [Nanontron](https://github.com/huggingface/nanotron) repository
+- [Ultrascale Playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook) describes how to efficiently scale training with Nanotron

docs/source/en/community_integrations/nemo_automodel_finetuning.md

@@ -0,0 +1,72 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# NeMo Automodel
+
+[NeMo Automodel](https://github.com/NVIDIA-NeMo/Automodel) is an open-source PyTorch DTensor-native training library from NVIDIA. It supports large and small scale pretraining and fine-tuning for [LLMs](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/llm.html) and [VLMs](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/vlm.html) for fast experimentation in research and production environments, with parallelism strategies including FSDP2, tensor, pipeline, expert, and context parallelism. For high throughput, it integrates kernels from DeepEP and TransformerEngine.
+
+Define your training run in a YAML config file (see full [config file](https://github.com/NVIDIA-NeMo/Automodel/blob/0d05e245e0bbc9128b869b21a3908512affc6cae/examples/llm_finetune/nemotron/nemotron_nano_v3_hellaswag_peft.yaml)).
+
+```yaml
+# Instantiate a Nemotron V3 Nano model
+model:
+  _target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
+  pretrained_model_name_or_path: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
+
+# Run SFT on HellaSwag
+dataset:
+  _target_: nemo_automodel.components.datasets.llm.hellaswag.HellaSwag
+  path_or_dataset: rowan/hellaswag
+  split: train
+
+# Train PEFT adapters
+peft:
+  _target_: nemo_automodel.components._peft.lora.PeftConfig
+  exclude_modules: ["*.out_proj"]  # mamba layers use custom kernels that take in the out_proj.weight directly, thus lora doesn't work here.
+  dim: 8
+  alpha: 32
+  use_triton: True
+
+# Use EP + FSDP2 for training
+distributed:
+  strategy: fsdp2
+  dp_size: none
+  tp_size: 1
+  cp_size: 1
+  ep_size: 4
+
+# ... other parameters
+```
+
+Launch training with `torchrun` using the command below.
+
+```bash
+torchrun -–nproc-per-node=4 examples/llm_finetune/finetune.py -c /path/to/yaml
+```
+
+## Transformers integration
+
+- Any LLM or VLM supported in Transformers can also be instantiated through NeMo Automodel. See the [full model coverage](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/overview.html).
+- Built on top of Hugging Face models with [`AutoModel.from_pretrained`], with dynamic high-performance layer swaps and support for more refined parallelisms like Expert Parallelism (EP).
+- Detects the architecture field in [`AutoConfig.from_pretrained`] to automatically load custom implementations like Nemotron Nano V3.
+- Follows the Transformers API closely for drop-in compatibility.
+
+## Resources
+
+- [NeMo Automodel](https://github.com/NVIDIA-NeMo/Automodel)
+- [NeMo Transformers API](https://docs.nvidia.com/nemo/automodel/latest/guides/huggingface-api-compatibility.html)
+- NeMo Automodel dense models and Mixture-of-Expert (MoE) [benchmarks](https://docs.nvidia.com/nemo/automodel/latest/performance-summary.html)
+- See the NeMo [pretraining](./nemo_automodel_pretraining) guide to learn how to use NeMo for pretraining

docs/source/en/community_integrations/nemo_automodel_pretraining.md

@@ -0,0 +1,75 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# NeMo Automodel
+
+[NeMo Automodel](https://github.com/NVIDIA-NeMo/Automodel) is an open-source PyTorch DTensor-native training library from NVIDIA. It supports large and small scale pretraining and fine-tuning for [LLMs](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/llm.html) and [VLMs](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/vlm.html) for fast experimentation in research and production environments, with parallelism strategies including FSDP2, tensor, pipeline, expert, and context parallelism. For high throughput, it integrates kernels from DeepEP and TransformerEngine. 
+
+```py
+# Instantiating Nemotron v3 Nano with expert parallelism, FSDP, and TransformerEngine + DeepEP kernels.
+import os
+
+import torch
+import torch.distributed as dist
+
+from nemo_automodel import NeMoAutoModelForCausalLM
+from nemo_automodel.recipes._dist_setup import setup_distributed
+
+dist.init_process_group(backend="nccl")
+torch.cuda.set_device(int(os.environ.get("LOCAL_RANK", 0)))
+torch.manual_seed(1111)
+
+dist_setup = setup_distributed(
+    {
+        "strategy": "fsdp2",
+        "dp_size": None,  # will be inferred from world_size and other parallelism sizes
+        "dp_replicate_size": None,
+        "tp_size": 1,
+        "pp_size": 1,
+        "cp_size": 1,
+        "ep_size": 8,
+    },
+    world_size=dist.get_world_size(),
+)
+kwargs = {
+    "device_mesh": dist_setup.device_mesh,
+    "moe_mesh": dist_setup.moe_mesh,
+    "distributed_config": dist_setup.strategy_config,
+    "moe_config": dist_setup.moe_config,
+}
+model = NeMoAutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16", **kwargs)
+print(model)
+dist.destroy_process_group()
+```
+Launch the script with `torchrun` using the command below.
+
+```bash
+torchrun --nproc-per-node=8 /path/to/script
+```
+
+## Transformers integration
+
+- Any LLM or VLM supported in Transformers can also be instantiated through NeMo Automodel. See the [full model coverage](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/overview.html).
+- Built on top of Hugging Face models with [`AutoModel.from_pretrained`], with dynamic high-performance layer swaps and support for more refined parallelisms like Expert Parallelism (EP).
+- Detects the architecture field in [`AutoConfig.from_pretrained`] to automatically load custom implementations like Nemotron Nano V3.
+- Follows the Transformers API closely for drop-in compatibility.
+
+## Resources
+
+- [NeMo Automodel](https://github.com/NVIDIA-NeMo/Automodel)
+- [NeMo Transformers API](https://docs.nvidia.com/nemo/automodel/latest/guides/huggingface-api-compatibility.html)
+- NeMo Automodel dense models and Mixture-of-Expert (MoE) [benchmarks](https://docs.nvidia.com/nemo/automodel/latest/performance-summary.html)
+- See the NeMo [fine-tuning](./nemo_automodel_finetuning) guide to learn how to use NeMo for fine-tuning

docs/source/en/community_integrations/sglang.md

@@ -0,0 +1,58 @@
+<!--Copyright 2025 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# SGLang
+
+[SGLang](https://docs.sglang.ai) is a low-latency, high-throughput inference engine for large language models (LLMs). It also includes a frontend language for building agentic workflows.
+
+Set `model_impl="transformers"` to load a Transformers modeling backend.
+
+```py
+import sglang as sgl
+
+llm = sgl.Engine("meta-llama/Llama-3.2-1B-Instruct", model_impl="transformers")
+print(llm.generate(["The capital of France is"], {"max_new_tokens": 20})[0])
+```
+
+Pass `--model-impl transformers` to the `sglang.launch_server` command for online serving.
+
+```bash
+python3 -m sglang.launch_server \
+  --model-path meta-llama/Llama-3.2-1B-Instruct \
+  --model-impl transformers \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+## Transformers integration
+
+Setting `model_impl="transformers"` tells SGLang to skip its native model matching and use the Transformers model directly.
+
+1. [`PreTrainedConfig.from_pretrained`] loads the model's `config.json` from the Hub or your Hugging Face cache.
+2. [`AutoModel.from_config`] resolves the model class based on the config.
+3. During loading, `_attn_implementation` is set to `"sglang"`. This routes attention calls through SGLang's RadixAttention kernels.
+4. SGLang's parallel linear class replaces linear layers to support tensor parallelism.
+5. The [load_weights](https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/models/transformers.py#L277) function populates the model with weights from safetensors files.
+
+The model benefits from all SGLang optimizations while using the Transformers model structure.
+
+> [!WARNING]
+> Compatible models require `_supports_attention_backend=True` so SGLang can control attention execution. See the [Building a compatible model backend for inference](./transformers_as_backend#model-implementation) guide for details.
+
+## Resources
+
+- [SGLang docs](https://docs.sglang.ai/supported_models/transformers_fallback.html) has more usage examples and tips for using Transformers as a backend.
+- [Transformers backend integration in SGLang](https://huggingface.co/blog/transformers-backend-sglang) blog post explains what this integration enables.

docs/source/en/community_integrations/tensorrt-llm.md

@@ -0,0 +1,41 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# TensorRT-LLM
+
+[TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) is optimizes LLM inference on NVIDIA GPUs. It compiles models into a TensorRT engine with in-flight batching, paged KV caching, and tensor parallelism. [AutoDeploy](https://nvidia.github.io/TensorRT-LLM/torch/auto_deploy/auto-deploy.html) accepts Transformers models without requiring any changes. It automatically converts the model to an optimized runtime.
+
+Pass a model id from the Hub to [build_and_run_ad.py](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/auto_deploy/build_and_run_ad.py) to run a Transformers model.
+
+```bash
+cd examples/auto_deploy
+python build_and_run_ad.py --model meta-llama/Llama-3.2-1B
+```
+
+Under the hood, AutoDeploy creates an [LLM](https://nvidia.github.io/TensorRT-LLM/llm-api/reference.html#tensorrt_llm.llmapi.LLM) class. It loads the model configuration with [`AutoConfig.from_pretrained`] and extracts any parallelism metadata stored in `tp_plan`. [`AutoModelForCausalLM.from_pretrained`] loads the model with the config and enables Transformers' built-in tensor parallelism.
+
+```py
+from tensorrt_llm._torch.auto_deploy import LLM
+
+llm = LLM(model="meta-llama/Llama-3.2-1B")
+```
+
+TensorRT-LLM extracts the model graph with `torch.export` and applies optimizations. It replaces Transformers attention with TensorRT-LLM [attention kernels](https://github.com/NVIDIA/TensorRT-LLM/tree/main/tensorrt_llm/_torch/attention_backend) and compiles the model into an optimized execution backend.
+
+## Resources
+
+- [TensorRT-LLM docs](https://nvidia.github.io/TensorRT-LLM/) for more detailed usage guides.
+- [AutoDeploy guide](https://nvidia.github.io/TensorRT-LLM/torch/auto_deploy/auto-deploy.html) explains how it works with advanced examples.

docs/source/en/community_integrations/torchtitan.md

@@ -0,0 +1,60 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# torchtitan
+
+[torchtitan](https://github.com/pytorch/torchtitan) is PyTorch's distributed training framework for large language models. It supports Fully Sharded Data Parallelism (FSDP), tensor, pipeline, and context parallelism (4D parallelism). torchtitan is fully compatible with [torch.compile](../perf_torch_compile), enabling kernel fusion and graph optimizations that significantly reduce memory overhead and speed up training.
+
+> [!NOTE]
+> Only dense models are supported at the moment.
+
+Use a Transformers model directly in torchtitan's distributed training infrastructure.
+
+```py
+import torch
+from torchtitan.config.job_config import JobConfig
+from torchtitan.experiments.transformers_modeling_backend.job_config import (
+    HFTransformers,
+)
+from torchtitan.experiments.transformers_modeling_backend.model.args import (
+    TitanDenseModelArgs,
+    HFTransformerModelArgs,
+)
+from torchtitan.experiments.transformers_modeling_backend.model.model import (
+    HFTransformerModel,
+)
+
+job_config = JobConfig()
+
+job_config.hf_transformers = HFTransformers(model="Qwen/Qwen2.5-7B")
+
+titan_args = TitanDenseModelArgs()
+model_args = HFTransformerModelArgs(titan_dense_args=titan_args).update_from_config(
+    job_config
+)
+
+model = HFTransformerModel(model_args)
+```
+
+## Transformers integration
+
+1. [`AutoConfig.from_pretrained`] loads the config for a given model. The config values are copied into torchtitan style args in `HFTransformerModelArgs`.
+2. torchtitan's `HFTransformerModel` wrapper scans the `architecture` field in the config and instantiates and loads the corresponding model class, like [`LlamaForCausalLM`].
+3. The `forward` path uses native Transformers components while leaning on torchtitan's parallelization and optimization methods. torchtitan treats the Transformers model as a torchtitan model without needing to rewrite anything.
+
+## Resources
+
+- [torchtitan](https://github.com/pytorch/torchtitan) repository 

docs/source/en/community_integrations/transformers_as_backend.md

@@ -0,0 +1,141 @@
+<!--Copyright 2025 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Building a compatible model backend for inference
+
+Transformers models are compatible with inference engines like [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://docs.sglang.ai). Use the same Transformers model anywhere and avoid reimplementing a model from scratch for each inference engine. This also helps with models that an engine does not natively reimplement, as long as the model supports the attention backend interface described below.
+
+This guide shows you how to implement a model in Transformers that works as a backend for any inference engine.
+
+## Model implementation
+
+1. Follow the model [contribution guidelines](./add_new_model) or the [custom model contribution guidelines](./custom_models). The model must have a valid `config.json` in its directory and a valid `auto_map` field pointing to the model class in the config.
+
+2. Use the [`AttentionInterface`] class for custom and optimized attention functions. This interface unlocks each inference engine's performance features. 
+
+   Use `ALL_ATTENTION_FUNCTIONS` when defining the attention layer and propagate `**kwargs` from the base `MyModel` class to the attention layers. Set `_supports_attention_backend` to `True` in [`PreTrainedModel`].
+   
+   Expand the code below for an example.
+
+    <details>
+    <summary>modeling_my_model.py</summary>
+
+    ```python
+    from transformers import PreTrainedModel
+    from transformers.modeling_utils import ALL_ATTENTION_FUNCTIONS
+    from torch import nn
+
+    class MyAttention(nn.Module):
+
+        def forward(self, hidden_states, **kwargs):
+            ...
+            attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]
+            attn_output, attn_weights = attention_interface(
+                self,
+                query_states,
+                key_states,
+                value_states,
+                **kwargs,
+            )
+            ...
+
+    class MyModel(PreTrainedModel):
+        _supports_attention_backend = True
+    ```
+
+    </details>
+
+3. Enable optional tensor or pipeline parallelism by adding the following keys to [`PreTrainedConfig`].
+
+    * `base_model_tp_plan` enables [tensor parallelism](./perf_infer_gpu_multi) by mapping fully qualified layer name patterns to tensor parallel styles. Supports only the `"colwise"` and `"rowwise"` partitioning strategies.
+    * `base_model_pp_plan` enables pipeline parallelism by mapping direct child layer names to tuples of lists of strings. The first element of the tuple contains the names of the input arguments. The last element contains the variable names of the layer outputs in the modeling code.
+
+    Expand the code below for an example.
+
+    <details>
+    <summary>configuration_my_model.py</summary>
+
+    ```python
+
+    from transformers import PreTrainedConfig
+
+    class MyConfig(PreTrainedConfig):
+        base_model_tp_plan = {
+            "layers.*.self_attn.k_proj": "colwise",
+            "layers.*.self_attn.v_proj": "colwise",
+            "layers.*.self_attn.o_proj": "rowwise",
+            "layers.*.mlp.gate_proj": "colwise",
+            "layers.*.mlp.up_proj": "colwise",
+            "layers.*.mlp.down_proj": "rowwise",
+        }
+        base_model_pp_plan = {
+            "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+            "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+            "norm": (["hidden_states"], ["hidden_states"]),
+        }
+    ```
+
+    </details>
+
+## Multimodal models
+
+Multimodal models require additional changes beyond the [vision language model contribution checklist](./contributing#vision-language-model-contribution-checklist). These changes ensure multimodal inputs are properly processed.
+
+1. The [`ProcessorMixin`] class must include the `self.image_token` and `self.image_token_ids` attributes. These placeholder tokens indicate image positions in the input. The same token appears in the input prompt for images and in the model code to scatter image features.
+
+2. The [`ProcessorMixin`] class must include a `self._get_num_multimodal_tokens` method. This method computes the number of placeholder tokens required for multimodal inputs with given sizes. It returns a [`MultiModalData`] object. Placeholders between `<image>` tokens, such as row or column tokens, don't count as image placeholders. Count only tokens replaced by image features later in the modeling code.
+
+3. The [`ProcessorMixin`] class must check the value of `return_mm_token_type_ids` and return `mm_token_type_ids`. This indicates whether each position is a text token (`0`), image placeholder token (`1`), or a video placeholder token (`2`). Multimodal token type id sequences must be contiguous with no breaks between consecutive tokens. Treat special tokens for beginning, ending, row, and column tokens as placeholders.
+
+Expand the code below for an example.
+
+<details>
+<summary>modeling_my_multimodal_model.py</summary>
+
+```python
+class MyMultimodalProcessor(ProcessorMixin):
+
+    def __call__(self, images=None, text=None, **kwargs):
+        if return_mm_token_type_ids:
+            mm_token_type_ids = np.zeros_like(input_ids)
+            mm_token_type_ids[input_ids == self.image_token_id] = 1
+            text_inputs["mm_token_type_ids"] = mm_token_type_ids.tolist()
+        return BatchFeature(data={**text_inputs, **image_inputs}, tensor_type=return_tensors)
+
+    def _get_num_multimodal_tokens(self, image_sizes=None, **kwargs):
+        """
+        Computes the number of placeholder tokens needed for multimodal inputs with the given sizes.
+        Args:
+            image_sizes (`list[list[int]]`, *optional*):
+                The input sizes formatted as (height, width) per each image.
+        Returns:
+            `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided
+            input modalities, along with other useful data.
+        """
+        vision_data = {}
+        if image_sizes is not None:
+            num_image_tokens = [256] * len(image_sizes) # 256 placeholder tokens for each image always
+            num_image_patches = [1] * len(image_sizes) # no patching, thus each image is processed as a single base image
+            vision_data.update({"num_image_tokens": num_image_tokens, "num_image_patches": num_image_patches})
+        return MultiModalData(**vision_data)
+```
+
+</details>
+
+## Resources
+
+* Read the [Transformers backend integration in vLLM](https://blog.vllm.ai/2025/04/11/transformers-backend.html) blog post for more details.
+* Read the [Transformers backend integration in SGLang](https://huggingface.co/blog/transformers-backend-sglang) blog post for more details.

docs/source/en/community_integrations/trl.md

@@ -0,0 +1,47 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# TRL
+
+[TRL](https://huggingface.co/docs/trl/index) is a post-training framework for foundation models. It includes methods like SFT, GRPO, and DPO. Each method has a dedicated trainer that builds on the [`Trainer`] class and scales from a single GPU to multi-node clusters.
+
+```py
+from datasets import load_dataset
+from trl import GRPOTrainer
+from trl.rewards import accuracy_reward
+
+dataset = load_dataset("trl-lib/DeepMath-103K", split="train")
+
+trainer = GRPOTrainer(
+    model="Qwen/Qwen2-0.5B-Instruct",
+    reward_funcs=accuracy_reward,
+    train_dataset=dataset,
+)
+trainer.train()
+```
+
+## Transformers integration
+
+TRL extends Transformers APIs and adds method-specific settings.
+
+- TRL trainers build on [`Trainer`]. Method-specific trainers like [`~trl.GRPOTrainer`] add generation, reward scoring, and loss computation. Config classes extend [`TrainingArguments`] with method-specific fields.
+
+- Model loading uses [`AutoConfig.from_pretrained`], then instantiates the model class from the config with that class' `from_pretrained`.
+
+## Resources
+
+- [TRL](https://huggingface.co/docs/trl/index) docs
+- [Fine Tuning with TRL](https://huggingface.co/datasets/trl-lib/documentation-images/resolve/main/Fine%20tuning%20with%20TRL%20(Oct%2025).pdf) talk

docs/source/en/community_integrations/unsloth.md

@@ -0,0 +1,70 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Unsloth
+
+[Unsloth](https://unsloth.ai/docs) is a fine-tuning and reinforcement framework that speeds up training and reduces memory usage for large language models. It supports training in 4-bit, 8-bit, and 16-bit precision with custom RoPE and Triton kernels. Unsloth works with Llama, Mistral, Gemma, Qwen, and other model families.
+
+```py
+from datasets import load_dataset
+from transformers import TrainingArguments
+from unsloth import FastLanguageModel
+from unsloth.trainer import UnslothTrainer
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="unsloth/Llama-3.2-1B-Instruct",
+    max_seq_length=2048,
+    load_in_4bit=True,
+)
+
+model = FastLanguageModel.get_peft_model(
+    model,
+    r=16,
+    lora_alpha=16,
+    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
+)
+
+dataset = load_dataset("trl-lib/Capybara", split="train[:500]")
+dataset = dataset.map(lambda x: {"text": x["conversations"][0]["value"]})
+
+trainer = UnslothTrainer(
+    model=model,
+    tokenizer=tokenizer,
+    train_dataset=dataset,
+    dataset_text_field="text",
+    max_seq_length=2048,
+    args=TrainingArguments(
+        output_dir="outputs",
+        per_device_train_batch_size=2,
+        num_train_epochs=1,
+    ),
+)
+
+trainer.train()
+```
+
+## Transformers integration
+
+Unsloth wraps Transformers APIs and patches internal methods for speed.
+
+- `FastLanguageModel.from_pretrained` loads config with [`AutoConfig.from_pretrained`]. It then loads a base model with [`AutoModelForCausalLM.from_pretrained`]. Before loading, Unsloth patches attention, decoder layer, and rotary embedding classes inside a Transformers model.
+
+- `UnslothTrainer` extends TRL's [`~trl.SFTTrainer`]. Unsloth patches [`~Trainer.compute_loss`] and [`~Trainer.training_step`] to fix gradient accumulation in older Transformers versions.
+
+## Resources
+
+- [Unsloth](https://unsloth.ai/docs) docs
+- [Make LLM Fine-tuning 2x faster with Unsloth and TRL](https://huggingface.co/blog/unsloth-trl) blog post

docs/source/en/community_integrations/vllm.md

@@ -0,0 +1,50 @@
+<!--Copyright 2025 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# vLLM
+
+[vLLM](https://github.com/vllm-project/vllm) is a high-throughput inference engine for serving LLMs at scale. It continuously batches requests and keeps KV cache memory compact with PagedAttention.
+
+Set `model_impl="transformers"` to load a model using the Transformers modeling backend.
+
+```py
+from vllm import LLM
+
+llm = LLM(model="meta-llama/Llama-3.2-1B", model_impl="transformers")
+print(llm.generate(["The capital of France is"]))
+```
+
+Pass `--model-impl transformers` to the `vllm serve` command for online serving.
+
+```bash
+vllm serve meta-llama/Llama-3.2-1B \
+    --task generate \
+    --model-impl transformers
+```
+
+## Transformers integration
+
+1. [`AutoConfig.from_pretrained`] loads the model's `config.json` from the Hub or your Hugging Face cache. vLLM checks the `architectures` field against its internal model registry to determine which vLLM model class to use.
+2. If the model isn't in the registry, vLLM calls [`AutoModel.from_config`] to load the Transformers model implementation instead.
+3. [`AutoTokenizer.from_pretrained`] loads the tokenizer files. vLLM caches some tokenizer internals to reduce overhead during inference.
+4. Model weights download from the Hub in safetensors format.
+
+Setting `model_impl="transformers"` bypasses the vLLM model registry and loads directly from Transformers. vLLM replaces most model modules (MoE, attention, linear layers) with its own optimized versions while keeping the Transformers model structure.
+
+## Resources
+
+- [vLLM docs](https://docs.vllm.ai/en/latest/models/supported_models.html#transformers) for more usage examples and tips.
+- [Integration with Hugging Face](https://docs.vllm.ai/en/latest/design/huggingface_integration/) explains how vLLM integrates with Transformers.