transformers/docs/source/en/model_doc/phi3.md

This model was published in HF papers on 2024-04-22 and contributed to Hugging Face Transformers on 2024-04-24.

Phi-3

Overview

The Phi-3 model was proposed in Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone by Microsoft.

Summary

The abstract from the Phi-3 paper is the following:

We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone. The innovation lies entirely in our dataset for training, a scaled-up version of the one used for phi-2, composed of heavily filtered web data and synthetic data. The model is also further aligned for robustness, safety, and chat format. We also provide some initial parameter-scaling results with a 7B and 14B models trained for 4.8T tokens, called phi-3-small and phi-3-medium, both significantly more capable than phi-3-mini (e.g., respectively 75% and 78% on MMLU, and 8.7 and 8.9 on MT-bench).

The original code for Phi-3 can be found here.

Usage tips

• This model is very similar to Llama with the main difference of [Phi3SuScaledRotaryEmbedding] and [Phi3YarnScaledRotaryEmbedding], where they are used to extend the context of the rotary embeddings. The query, key and values are fused, and the MLP's up and gate projection layers are also fused.
• The tokenizer used for this model is identical to the [LlamaTokenizer], with the exception of additional tokens.

How to use Phi-3

Phi-3 has been integrated in the development version (4.40.0.dev) of transformers. Until the official version is released through pip, ensure that you are doing one of the following:

• When loading the model, ensure that trust_remote_code=True is passed as an argument of the from_pretrained() function.

• Update your local transformers to the development version: pip uninstall -y transformers && pip install git+https://github.com/huggingface/transformers. The previous command is an alternative to cloning and installing from the source.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

messages = [{"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)

outputs = model.generate(inputs, max_new_tokens=32)
text = tokenizer.batch_decode(outputs)[0]
print(text)
<|user|> Can you provide ways to eat combinations of bananas and dragonfruits?<|end|><|assistant|> Certainly! Bananas and dragonfruits can be combined in various delicious ways. Here are some creative ideas for incorporating both fruits

Phi3Config

autodoc Phi3Config

Phi3Model

autodoc Phi3Model - forward

Phi3ForCausalLM

autodoc Phi3ForCausalLM - forward - generate

Phi3ForSequenceClassification

autodoc Phi3ForSequenceClassification - forward

Phi3ForTokenClassification

autodoc Phi3ForTokenClassification - forward