• Supported service: llm
  • Key: groq
  • Integrated: No. See BYO Keys for more details.

​
Service options

​
model
string

The model that will complete your prompt. Available models can be found here.

{
  "service_options": {
    "groq": {
      "model": "llama-3.1-70b-versatile"
    }
  }
}

​
Configuration options

​
model
string

The model that will complete your prompt. Available models can be found here.

{
  "name": "model",
  "value": "llama-3.1-70b-versatile"
}
​
extra
object

A dictionary that can contain any additional parameters supported by Groq that you want to pass to the API. Refer to the Groq docs for more information on each of these configuration options.

{
  "name": "extra",
  "value": {
    "temperature": 0.9,
    "max_tokens": 2048,
    "top_p": 0.5
  }
}

​
Function calling

Many of Groq’s models, including their versions of Llama 3 and 3.1, support function calling using the OpenAI interface. For examples of how to use that approach, see the OpenAI service page.

Alternatively, Groq also supports Llama 3.1’s function calling interface. Llama 3.1’s function calling documentation is located here. Llama doesn’t use OpenAI-style function calling; instead, you need to add to your system prompt to enable the LLM to use functions. If you use the following recommended format, Daily Bots will be able to detect function call responses from the LLM and allow you to follow the directions in the function calling tutorial.

First, define a weatherTool object, and/or any other functions you want to call:

const weatherTool = {
  name: "get_current_weather",
  description: "Get the current weather in a given location",
  parameters: {
    type: "object",
    properties: {
      location: {
        type: "string",
        description: "The city and state, e.g. San Francisco, CA",
      },
    },
    required: ["location"],
  },
};

Then, reference that weatherTool object in your system prompt:

export const defaultConfig = [
  {
    service: "llm",
    options: [
      {
        name: "initial_messages",
        value: [
          {
            role: "user",
            content: `
              You have access to the following functions:

              Use the function '${weatherTool["name"]}' to '${
              weatherTool["description"]
            }':
              ${json.stringify(weatherTool)}

              If you choose to call a function ONLY reply in the following format with no prefix or suffix:

              <function=example_function_name>{{\"example_name\": \"example_value\"}}</function>

              Reminder:
              - Function calls MUST follow the specified format, start with <function= and end with </function>
              - Required parameters MUST be specified
              - Only call one function at a time
              - Put the entire function call reply on one line
              - If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls

              You are a TV weatherman named Dallas Storms. Your job is to present the weather to me. You can call the 'get_weather' function to get weather information. Start by asking me for my location. Then, use 'get_weather' to give me a forecast. Then, answer any questions I have about the weather. Keep your introduction and responses very brief. You don't need to tell me if you're going to call a function; just do it directly. Keep your words to a minimum. When you're delivering the forecast, you can use more words and personality.`,
          },
        ],
      },
      { name: "run_on_config", value: true },
    ],
  },
];
GrokOpenAI