Released Models Version <1.0.0-alpha> (03/08/23)

🇹🇭 OpenThaiGPT 1.0.0-alpha (3 August 2023)

🇹🇭 OpenThaiGPT Version 1.0.0-alpha is the first Thai implementation of a 7B-parameter LLaMA v2 Chat model finetuned to follow Thai translated instructions and makes use of the Huggingface LLaMA implementation.

Web Demo:

Colab Demo:

Change Logs

🇹🇭 Version 1.0.0-alpha (Facebook LLama V2 Model)

Release date: 3 August 2023

🇹🇭 OpenThaiGPT Version 1.0.0-alpha is the first Thai implementation of a 7B-parameter LLaMA v2 Chat model finetuned to follow Thai translated instructions and makes use of the Huggingface LLaMA implementation.

Changes

(1) Using Facebook LLama v2 model 7b chat as a base model which is pretrained on over 2 trillion token. (2) Context Length is upgrade from 2048 token to 4096 token (3) Allow research and commerical use.

License

Source Code: License Apache Software License 2.0. Weight: Research and commercial uses.

Code and Weight

Colab Demo: Finetune Code: (Same code as OpenThaiGPT 0.1.0-beta) Inference Library: Weight (Lora Adapter): Weight (Huggingface Checkpoint):

Authors

• Kobkrit Viriyayudhakorn ([email protected])

• Sumeth Yuenyong ([email protected])

• Thaweewat Rugsujarit ([email protected])

• Jillaphat Jaroenkantasima ([email protected])

---

Version 0.1.0-beta (Facebook LLama Model)

Release date: 16 May 2023

OpenThaiGPT Version 0.1.0-beta is a 7B-parameter LLaMA model finetuned to follow Thai translated instructions below and makes use of the Huggingface LLaMA implementation.

Statistics

Number of parameters: 7B Dimension: 4096 Context Length: 2048 n heads: 32 n layers: 32 n tokens: 1T

License

Source Code: License Apache Software License 2.0. Weight: For research use only (due to the Facebook LLama's Weight LICENSE). Note that: A commercial use license for OpenThaiGPT 0.1.0 weight will be released later soon!

Code and Weight

Finetune Code: Inference Library: Weight (Lora Adapter):

Authors

Kobkrit Viriyayudhakorn ([email protected]), Sumeth Yuenyong ([email protected]) and Thaweewat Rugsujarit ([email protected]).

Trained Datasets

---

Version 0.1.0-alpha (ByT5-XL Model)

Release date: 24 April 2023 PoC Testing Website: Model and Weight: PIP Installation Page: Code Example: ----

OpenThaiGPT version 0.1.0-alpha

Thai First 3 billion params models

• First Thai Byte-Level Text-to-Text Transfer Transformer

• Support Instruction following

  • Translation to Thai

  • Explanation

PoC Version 0.0.4 (The Fourth PoC Version)

Release date: 12 March 2023 PoC Testing Website: Model and Weight: PIP Installation Page: Code Example: ----

OpenThaiGPT version 0.0.4

The Fourth PoC Model

• ตอบคำถามได้ลงรายละเอียดมากขึ้น และตอบคำถามได้ดีขึ้นกว่า 0.0.3 เป็นส่วนมาก

• Pretraining Model: GPT-2 Thai-base

• InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 12,920 Thai InstructGPT

• RLHF: None

PoC Version 0.0.3 (The Third PoC Version)

Release date: 28 February 2023 Model and Weight: PIP Installation Page: Code Example: ----

OpenThaiGPT version 0.0.3

The Third PoC Model

• Pretraining Model: GPT-2 Thai-base

• InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 7,000 Thai InstructGPT

• RLHF: None

• Developer: Kobkrit Viriyayudhakorn ([email protected])

PoC Version 0.0.2 (The Second PoC Version)

Release date: 27 February 2023 Model and Weight: PIP Installation Page: {Coming Soon} Colab Example: {Coming Soon} ----

OpenThaiGPT version 0.0.2

The Second PoC Model

• Pretraining Model: GPT-2 Thai-base

• InstructDataset: 7,000 Thai InstructGPT

• RLHF: None

Developer: Kobkrit Viriyayudhakorn ([email protected])

PoC Version 0.0.1 (Very First PoC Version)

Release date: 20 February 2023 Model and Weight: PIP Installation Page: {Coming Soon} Colab Example: {Coming Soon} ----

The Very First PoC Model

• Pretraining Model: GPT-2 Thai-base

• InstructDataset: 298,678 QA Pairs getting from 70,000 Pantip katoos + Wikipedia QA by iApp

• RLHF: None

• Developer: Kobkrit Viriyayudhakorn ([email protected])