Enhanced Messaging Types Support

Background:

Our current product lacks the support for System Messages in ChatGPT/Llama2 and requires a custom instruction mechanism [INST]<<SYS>><</SYS>>[/INST] for Llama2. The absence of a structured messaging type framework restricts the flexibility and interaction model that can be crucial for developer-centric applications.

Site like ChatGPT Playground https://platform.openai.com/playground

Benefits:

• Improved interaction: This message types allow more interactive and intuitive communication model

• Better UX: For Llama2 user, user dont need to type in [INST]<<SYS>> any more, also allow user to swap LLM easily

• Future-proofing: ChatGPT playground already shown usecases

Proposal:

1. Support Multiple message and Message Type like System, User, Assistant for both ChatGPT and Llama2

2. Support extendable API for message type to allow conversational usecase, useful for things like ChatBot / Interview AI

{
  "input": {
    "question":"USA"
  },
  "messages": [{
    "assistant": "USA is a ..."
  }, {
    "user": "How about Mexico?"
  }]
}

Technical

• ChatGPT

From ChatGPT-3.5-turbo onwards they are supporting multiple messages

      {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the world series in 2020?"},
        {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
        {"role": "user", "content": "Where was it played?"}

https://platform.openai.com/docs/guides/gpt/chat-completions-api

• Llama2

  

Consideration / Open Questions

• How to handle maximum token size?

• Advance user want to customize Llama2, how can they do it?