06f1fd69a6
5.0 KiB
This model was contributed to Hugging Face Transformers on 2022-09-14.
</div>
GPT-NeoX-Japanese
GPT-NeoX-Japanese, a Japanese language model based on GPT-NeoX. Japanese uses three types of characters (hiragana, katakana, kanji) and has a huge vocabulary. This model uses BPEEncoder V2, a sub-word tokenizer to handle the different characters.
The model also removes some bias parameters for better performance.
You can find all the original GPT-NeoX-Japanese checkpoints under the ABEJA organization.
Tip
This model was contributed by Shinya Otani, Takayoshi Makabe, Anuj Arora, and Kyo Hattori from ABEJA, Inc..
Click on the GPT-NeoX-Japanese models in the right sidebar for more examples of how to apply GPT-NeoX-Japanese to different language tasks.
The example below demonstrates how to generate text with [Pipeline] or the [AutoModel], and from the command line.
from transformers import pipeline
pipeline = pipeline(task="text-generation",
model="abeja/gpt-neox-japanese-2.7b", device=0)
pipeline("人とAIが協調するためには、")
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("abeja/gpt-neox-japanese-2.7b", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
input_ids = tokenizer("人とAIが協調するためには、", return_tensors="pt").input_ids.to(model.device)
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Quantization reduces the memory burden of large models by representing the weights in a lower precision. Refer to the Quantization overview for more available quantization backends.
The example below uses bitsandbytes to only quantize the weights to 4-bits.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype="float16"
)
model = AutoModelForCausalLM.from_pretrained(
"abeja/gpt-neox-japanese-2.7b",
quantization_config=quantization_config,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("abeja/gpt-neox-japanese-2.7b")
input_ids = tokenizer.encode("人とAIが協調するためには、", return_tensors="pt").to(model.device)
output = model.generate(input_ids)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Use the AttentionMaskVisualizer to better understand what tokens the model can and cannot attend to.
from transformers.utils.attention_visualizer import AttentionMaskVisualizer
visualizer = AttentionMaskVisualizer("abeja/gpt-neox-japanese-2.7b")
visualizer("<img>What is shown in this image?")
Resources
Refer to the Training a better GPT model: Learnings from PaLM blog post for more details about how ABEJA trained GPT-NeoX-Japanese.
GPTNeoXJapaneseConfig
autodoc GPTNeoXJapaneseConfig
GPTNeoXJapaneseTokenizer
autodoc GPTNeoXJapaneseTokenizer
GPTNeoXJapaneseModel
autodoc GPTNeoXJapaneseModel - forward
GPTNeoXJapaneseForCausalLM
autodoc GPTNeoXJapaneseForCausalLM - forward