import openai# Configure OpenAI client to use vLLM serveropenai.api_base ="http://127.0.0.1:8000/v1"openai.api_key ="dummy"# vLLM doesn't require a real API keyprompt = "<|im_start|>system\nคุณคือผู้ช่วยตอบคำถามที่ฉลาดและซื่อสัตย์<|im_end|>\n<|im_start|>user\nกรุงเทพมหานครคืออะไร<|im_end|>\n<|im_start|>assistant\n"
try: response = openai.Completion.create( model=".", # Specify the model you're using with vLLM prompt=prompt, max_tokens=512, temperature=0.7, top_p=0.8, top_k=40, stop=["<|im_end|>"] )print("Generated Text:", response.choices[0].text)exceptExceptionas e:print("Error:", str(e))
The current config.json is set for context length up to 32,768 tokens. To handle extensive inputs exceeding 32,768 tokens, we utilize YaRN, a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
For supported frameworks, you could add the following to config.json to enable YaRN:
The Tool Calling feature in OpenThaiGPT 1.5 enables users to efficiently call various functions through intelligent responses. This includes making external API calls to retrieve real-time data, such as current temperature information, or predicting future data simply by submitting a query.
For example, a user can ask OpenThaiGPT, “What is the current temperature in San Francisco?” and the AI will execute a pre-defined function to provide an immediate response without the need for additional coding.
This feature also allows for broader applications with external data sources, including the ability to call APIs for services such as weather updates, stock market information, or data from within the user’s own system.
Example:
import openaidefget_temperature(location,date=None,unit="celsius"):"""Get temperature for a location (current or specific date)."""if date:return{"temperature":25.9,"location": location,"date": date,"unit": unit}return{"temperature":26.1,"location": location,"unit": unit}tools = [{"name":"get_temperature","description":"Get temperature for a location (current or by date).","parameters":{"location":"string","date":"string (optional)","unit":"enum [celsius, fahrenheit]"},}]messages = [{"role":"user","content":"อุณหภูมิที่ San Francisco วันนี้ีและพรุ้่งนี้คือเท่าไร่?"}]# Simulated response flow using OpenThaiGPT Tool Callingresponse = openai.ChatCompletion.create( model=".", messages=messages, tools=tools, temperature=0.7, max_tokens=512)print(response)
If OpenThaiGPT has been beneficial for your work, kindly consider citing it as follows:
Bibtex
@misc{yuenyong2024openthaigpt15thaicentricopen,title={OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Model}, author={Sumeth Yuenyong and Kobkrit Viriyayudhakorn and Apivadee Piyatumrong and Jillaphat Jaroenkantasima},year={2024},eprint={2411.07238},archivePrefix={arXiv},primaryClass={cs.CL},url={https://arxiv.org/abs/2411.07238}, }
APA Style (for TXT, MS Word)
Yuenyong, S., Viriyayudhakorn, K., Piyatumrong, A., & Jaroenkantasima, J. (2024). OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Model. arXiv [Cs.CL]. Retrieved from http://arxiv.org/abs/2411.07238
Disclaimer: Provided responses are not guaranteed.
