transformers/docs/source/en/model_doc/hyperclovax.md

<<<<<<< Updated upstream This model was released on 2025-07-21 and added to Hugging Face Transformers on 2026-05-08.

This model was contributed to Hugging Face Transformers on 2026-05-08.

Stashed changes

HyperCLOVA X

Overview

HyperCLOVA X SEED Think is NAVER Cloud's language model combining pruning and knowledge distillation with advanced reasoning capabilities. The 14B model features a Transformer-based architecture with Peri-Layer Normalization and Maximal Update Parameterization (μP), 14.74B parameters, and 32k context length. It supports dual-mode reasoning (think / non-think) and function calling via a ChatML-based format.

The model was trained with a multi-stage RL pipeline (SFT → RLVR → Length Controllability → joint RLHF+RLVR) and achieves strong performance on Korean language benchmarks and reasoning tasks.

HyperCLOVA X shares a high degree of implementation similarity with Granite, with the following modifications:

• Maximal Update Parametrization (MuP): uses per-config scaling factors (attention_multiplier, residual_multiplier, embedding_multiplier, logits_scaling) to enable stable training across model sizes. head_dim (defaults to hidden_size // num_attention_heads) is used to compute the default attention_multiplier.
• Peri-Layer Normalization (optional): applies an extra RMSNorm after each sub-layer output when use_post_norm=True.

This model was contributed by NAVER Cloud HyperCLOVA X Team. The original model can be found at naver-hyperclovax/HyperCLOVAX-SEED-Think-14B.

Usage

The model uses a ChatML-based format with special tokens <|im_start|>, <|im_end|>, <|endofturn|>, and <|stop|>. The apply_chat_template method accepts the following kwargs:

• force_reasoning=True — always think before answering
• skip_reasoning=True — always answer directly (non-think mode)
• Default (None) — model decides based on context

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of South Korea?"},
]
# Pass force_reasoning=True to always think, or skip_reasoning=True to skip thinking.
model_inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    # force_reasoning=True,
    # skip_reasoning=True,
).to(model.device)

output = model.generate(
    **model_inputs,
    tokenizer=tokenizer,
)
print(tokenizer.decode(output[0][model_inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

HyperCLOVAXConfig

autodoc HyperCLOVAXConfig

HyperCLOVAXModel

autodoc HyperCLOVAXModel - forward

HyperCLOVAXForCausalLM

autodoc HyperCLOVAXForCausalLM - forward