first commit

docs/source/en/serve-cli/cursor.md

@@ -0,0 +1,46 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Cursor
+
+[Cursor](https://cursor.com) is an AI-powered code editor that offers models like Sonnet, Gemini, and GPT. Enable [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS) to serve local models (it's disabled by default for security).
+
+```bash
+transformers serve --enable-cors
+```
+
+Cursor requires a public address to reach your local server. Use [ngrok](https://ngrok.com/) to create a tunnel. Sign up, authenticate, and run this command.
+
+```bash
+ngrok http localhost:8000
+```
+
+<h3 align="center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/transformers_serve_ngrok.png"/>
+</h3>
+
+Open Cursor and go to **Settings > Cursor Settings > Models > API Keys**.
+
+1. Deselect all models in the list.
+2. Add your desired model (Qwen/Qwen3-4B).
+3. Enter any text in the **OpenAI API Key** field (required).
+4. Paste the ngrok Forwarding address into **Override OpenAI Base URL**. Append `/v1` to the URL (https://...ngrok-free.app/v1).
+5. Click **Verify**.
+
+Your model is ready to use.
+
+<h3 align="center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/transformers_serve_cursor.png"/>
+</h3>

docs/source/en/serve-cli/jan.md

@@ -0,0 +1,29 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Jan
+
+[Jan](https://jan.ai/) is an open-source ChatGPT alternative. It supports `transformers serve` natively without extra tunneling.
+
+Go to **Settings > Model Providers**.
+
+1. Set **Base URL** to http://localhost:8000/v1.
+2. Enter any text in **API Key** (required).
+
+Check the **Models** section (click **Refresh** if empty). Click **+** to add a specific model like Qwen/Qwen3-4B.
+
+<h3 align="center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/transformers_serve_jan_model_providers.png"/>
+</h3>

docs/source/en/serve-cli/openweb_ui.md

@@ -0,0 +1,34 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Open WebUI
+
+[Open WebUI](https://openwebui.com/) provides a self-hosted interface for offline models. Enable [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS) to serve local models (it's disabled by default for security).
+
+```bash
+transformers serve --enable-cors
+```
+
+To use the server for speech-to-text, go to **Settings > Audio**.
+
+1. Set **Speech-to-Text Engine** to **OpenAI**.
+2. Set the URL to http://localhost:8000/v1.
+3. Enter a transcription model in **SST Model** (openai/whisper-large-v3). Find more models on the [Hub](https://huggingface.co/models?pipeline_tag=automatic-speech-recognition&sort=trending).
+
+<h3 align="center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/transformers_openwebui_stt_settings.png"/>
+</h3>
+
+Start a chat and speak. The model transcribes your audio into the input field.

docs/source/en/serve-cli/serving_optims.md

@@ -0,0 +1,85 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# Server optimizations
+
+`transformers serve` includes optimizations to improve throughput and reduce memory usage.
+
+## Continuous batching
+
+[Continuous batching](../continuous_batching) dynamically groups and interleaves requests to share forward passes on the GPU. New requests join the batch as others progress through prefill. Completed requests drop out after decoding. This increases GPU utilization and throughput without compromising latency.
+
+Add `--continuous-batching` to enable it.
+
+```sh
+transformers serve \
+  --continuous-batching \
+  --attn-implementation "sdpa"
+```
+
+## Quantization
+
+[Quantization](../quantization/overview) reduces memory usage by mapping weights to a lower precision. `transformers serve` is compatible with all quantization methods in Transformers. It supports pre-quantized models and runtime quantization.
+
+Pre-quantized models don't require any changes. They offer the best balance between performance and accuracy. Install the appropriate quantization library. Then pass the pre-quantized model from the Hub to the `model` argument.
+
+```sh
+curl http://localhost:8000/v1/responses \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "Qwen/Qwen3-8B-GGUF",
+    "stream": true,
+    "input": "Tell me a three sentence bedtime story about a unicorn."
+  }'
+```
+
+Use `--quantization` to quantize a model at runtime. This is useful for new checkpoints or finetunes without pre-quantized weights. Only [bitsandbytes](../quantization/bitsandbytes) 4-bit and 8-bit quantization are supported.
+
+```sh
+transformers serve \
+  --quantization bnb-4bit
+```
+
+## Attention backend
+
+An optimized [attention backend](../attention_interface) improves memory efficiency and speeds up inference.
+
+```sh
+transformers serve \
+  --continuous-batching \
+  --attn-implementation "flash_attention_2"
+```
+
+## Compile
+
+[torch.compile](../perf_torch_compile) traces and compiles the decode loop for faster inference.
+
+> [!NOTE]
+> Compile is incompatible with continuous batching.
+
+```sh
+transformers serve \
+  --compile
+```
+
+## Data type
+
+The `"bfloat16"` or `"float16"` [data types](../models#model-data-type) save memory and increase throughput.
+
+```sh
+transformers serve \
+  --continuous-batching \
+  --dtype "bfloat16"
+```

docs/source/en/serve-cli/tiny_agents.md

@@ -0,0 +1,56 @@
+<!--Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+
+# tiny-agents
+
+[tiny-agents](https://huggingface.co/blog/python-tiny-agents) offers a minimal pattern for building tool-using agents. A small Python loop connects an MCP client to tools from MCP servers. Hugging Face Spaces work as MCP servers. Find more compatible Spaces on the [Hub](https://huggingface.co/spaces?filter=mcp-server).
+
+Create a config file that points to your local model and the tool server. This example uses an image generation [Space](https://evalstate-flux1-schnell.hf.space/).
+
+```json
+{
+    "model": "Menlo/Jan-nano",
+    "endpointUrl": "http://localhost:8000",
+    "servers": [
+        {
+            "type": "sse",
+            "url": "https://evalstate-flux1-schnell.hf.space/gradio_api/mcp/sse"
+        }
+    ]
+}
+```
+
+Run the agent with your config.
+
+```bash
+tiny-agents run path/to/your/config.json
+```
+
+Ensure `transformers serve` is running in the background. The agent will use your local model to run the image generation tool.
+
+```bash
+Agent loaded with 1 tools:
+ • flux1_schnell_infer
+»  Generate an image of a cat on the moon
+<Tool req_0_tool_call>flux1_schnell_infer {"prompt": "a cat on the moon", "seed": 42, "randomize_seed": true, "width": 1024, "height": 1024, "num_inference_steps": 4}
+
+Tool req_0_tool_call
+[Binary Content: Image image/webp, 57732 bytes]
+The task is complete and the content accessible to the User
+Image URL: https://evalstate-flux1-schnell.hf.space/gradio_api/file=/tmp/gradio/3dbddc0e53b5a865ed56a4e3dbdd30f3f61cf3b8aabf1b456f43e5241bd968b8/image.webp
+380576952
+
+I have generated an image of a cat on the moon using the Flux 1 Schnell Image Generator. The image is 1024x1024 pixels and was created with 4 inference steps. Let me know if you would like to make any changes or need further assistance!
+```