Skip to content

Generative AI

Adds a message to a Conversation, that the language model will begin replying to. You can receive the reply one piece at a time by calling conversation_get_reply_piece(conversation c) in a loop

Parameters:

NameTypeDescription
cConversationThe Conversation object to check
messageStringThe user message to add to the conversation - the language model will reply to this

Signatures:

void conversation_add_message(conversation c, const string &message)

Returns a reply from a Conversation, without any related thoughts.

Parameters:

NameTypeDescription
convConversationThe Conversation to recieve the reply from

Return Type: String

Returns: The response from the model

Signatures:

string conversation_get_reply(conversation conv)

Returns a reply from a Conversation, with the ability to indicate if thoughts should be included.

Parameters:

NameTypeDescription
convConversationThe Conversation to recieve the reply from
with_thoughtsBooleanA boolean to indicate if thoughts should be included in the reply

Return Type: String

Returns: The response from the model

Signatures:

string conversation_get_reply(conversation conv, bool with_thoughts)

Returns a single piece of a reply (generally one word at a time) from the Conversation You can use a loop while checking Conversation Is Replying to retrieve the reply as it generates

Parameters:

NameTypeDescription
cConversationThe Conversation object to recieve the reply from

Return Type: String

Returns: Returns a small piece of the reply (generally 1 word or less)

Signatures:

string conversation_get_reply_piece(conversation c)

Checks if a language model is currently generating a reply within a Conversation. If so, you can continue to receive the message with conversation_get_reply_piece(conversation c)

Parameters:

NameTypeDescription
cConversationThe Conversation object to check

Return Type: Boolean

Returns: Returns whether the language model is still generating a reply

Signatures:

bool conversation_is_replying(conversation c)

Checks if a language model is currently “thinking” while generating a reply within a Conversation. You can use this to filter out the “thoughts” and display them differently (or hide them entirely)

Parameters:

NameTypeDescription
cConversationThe Conversation object to check

Return Type: Boolean

Returns: Returns whether the language model is currently thinking while generating a reply

Signatures:

bool conversation_is_thinking(conversation c)

Creates a new Conversation object, that uses the default language model. The Conversation object can have messages added to it, and responses streamed back from it via the other Conversation functions and procedures

Return Type: Conversation

Returns: Returns a new Conversation object.

Signatures:

conversation create_conversation()

Creates a new Conversation object, that uses a chosen language model. The Conversation object can have messages added to it, and responses streamed back from it via the other Conversation functions and procedures

Parameters:

NameTypeDescription
modelLanguage ModelThe language model to use

Return Type: Conversation

Returns: Returns a new Conversation object.

Signatures:

conversation create_conversation(language_model model)

Releases all of the Conversation objects which have been loaded.

Signatures:

void free_all_conversations()

Frees the resources associated with the Conversation object.

Parameters:

NameTypeDescription
cConversationThe Conversation object whose resources should be released.

Signatures:

void free_conversation(conversation c)

Generates a reply to a textual prompt by a language model The language model will respond to the textual prompt in a chat style format. It will follow instructions and answer questions. Instruct or Thinking models are recommended. Base models likely won’t output sensible results.

Parameters:

NameTypeDescription
modelLanguage ModelThe language model to use
promptStringThe prompt for the language model to reply to.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_reply(language_model model, string prompt)

Generates a reply to a textual prompt by a language model The language model will respond to the textual prompt in a chat style format. It will follow instructions and answer questions. Instruct or Thinking models are recommended. Base models likely won’t output sensible results.

Parameters:

NameTypeDescription
promptStringThe prompt for the language model to reply to.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_reply(string prompt)

Generates text that continues from a prompt, with a maximum of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.

Parameters:

NameTypeDescription
modelLanguage ModelThe language model to use
textStringThe input text for the language model to continue.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_text(language_model model, string text)

Generates text that continues from a prompt, with a maximum of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.

Parameters:

NameTypeDescription
modelLanguage ModelThe language model to use
textStringThe input text for the language model to continue.
max_tokensIntegerThe maximum tokens used in response - determining the length of the output and the time taken. Keep this small for reasonable execution times.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_text(language_model model, string text, int max_tokens)

Generates text that continues from a prompt - with default of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.

Parameters:

NameTypeDescription
textStringThe input text for the language model to continue.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_text(string text)

Generates text that continues from a prompt. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.

Parameters:

NameTypeDescription
textStringThe input text for the language model to continue.
max_tokensIntegerThe maximum tokens used in response - determining the length of the output and the time taken. Keep this small for reasonable execution times.

Return Type: String

Returns: The generated reply.

Signatures:

string generate_text(string text, int max_tokens)

The Conversation type is used to refer to conversations between the user and a language model. You can use it to send messages to the language model, and stream responses back.

All Conversation objects are:

  • created with create_conversation(), create_conversation(language_model model)

  • and must be released using free_conversation() (to release a specific Conversation object) or free_all_conversation() (to release all created Conversation objects).


Language Models: Choose between different language models to trade off speed and intelligence Each model is scaled to fit within 1~2GB and will be automatically downloaded when needed - feel free to try them out!

ConstantDescription
QWEN3_0_6B_BASEQwen3 0.6B Base model - small, extremely fast and good for text completion. Very limited world knowledge.
QWEN3_0_6B_INSTRUCTQwen3 0.6B Instruct model (default) - small, extremely fast and can follow simple instructions. Very limited world knowledge.
QWEN3_0_6B_THINKINGQwen3 0.6B Thinking model - small, extremely fast and can follow more specific instructions, but has a short delay before starting to reply. Very limited world knowledge.
QWEN3_1_7B_BASEQwen3 1.7B Base model - decently fast and good for text completion. Limited world knowledge.
QWEN3_1_7B_INSTRUCTQwen3 1.7B Instruct model - decently fast and can follow instructions. Limited world knowledge.
QWEN3_1_7B_THINKINGQwen3 1.7B Thinking model - decently fast and can follow more difficult instructions, but has a delay before starting to reply. Limited world knowledge.
QWEN3_4B_BASEQwen3 4B Base model - slower but excellent for text completion/pattern based completion
QWEN3_4B_INSTRUCTQwen3 4B Instruct model - slower but can follow complex instructions
QWEN3_4B_THINKINGQwen3 4B Thinking model - slower but can follow complex and specific instructions, but has a potentially long delay before starting to reply
GEMMA3_270M_BASEGemma3 270M Base model - tiny, extremely fast, and good for text completion. Very limited world knowledge.
GEMMA3_270M_INSTRUCTGemma3 270M Instruct model - tiny, extremely fast, and good for very simple instructions. Very limited world knowledge.
GEMMA3_1B_BASEGemma3 1B Base model - fast and good for text completion. Has decent world knowledge and multilingual abilities.
GEMMA3_1B_INSTRUCTGemma3 1B Instruct model - fast and can follow instructions. Has decent world knowledge and multilingual abilities.
GEMMA3_4B_BASEGemma3 4B Base model - slower but good for text completion/pattern based completion. Has decent world knowledge and multilingual abilities.
GEMMA3_4B_INSTRUCTGemma3 4B Instruct model - slower but can follow complex instructions. Has decent world knowledge and multilingual abilities.