LLM function calling workflows (Part 4, Universal specs)

Introduction

This blog post (notebook) shows how to utilize Large Language Model (LLM) Function Calling with the Raku package “LLM::Functions”, [AAp1].

“LLM::Functions” supports high level LLM function calling via llm-synthesize and llm-synthesize-with-tools. (The latter provides more options for the tool invocation process like max-iterations or overriding tool specs.)

At this point “LLM::Functions” supports function calling in the styles of OpenAI’s ChatGPT and Google’s Gemini. If the LLM configuration is not set with the names “ChatGPT” or “Gemini”, then the function calling style used is that of ChatGPT. (Many LLM providers — other than OpenAI and Gemini — tend to adhere to OpenAI’s API.)

Remark: LLM “function calling” is also known as LLM “tools” or “LLM tool invocation.”

In this document, non-trivial Stoichiometry computations are done with the Raku package “Chemistry::Stoichiometry”, [AAp4]. Related plots are done with the Raku package “JavaScript::D3”, [AAp6].

Big picture

Inversion of control is a way to characterize LLM function calling. This means the LLM invokes functions or subroutines that operate on an external system, such as a local computer, rather than within the LLM provider’s environment. See the section “Outline of the overall process” of “LLM function calling workflows (Part 1, OpenAI)”, [AA1].

Remark: The following Software Framework building principles (or mnemonic slogans) apply to LLM function calling:

“Don’t call us, we’ll call you.” (The Hollywood Principle)
“Leave the driving to us.” (Greyhound Lines, Inc.)

The whole series

This document is the fourth of the LLM function calling series, [AA1 ÷ AA4]. The other three show lower-level LLM function calling workflows.

Here are all blog posts of the series:

Overall comments and observations

Raku’s constellation of LLM packages was behind with the LLM tools.
- There are two main reasons for this:
  - For a long period of time (say, 2023 & 2024) LLM tool invocation was unreliable.
    - Meaning, tools were invoked (or not) in an unexpected manner.
  - Different LLM providers use similar but different protocols for LLM tooling.
    - And that poses “interesting” development choices. (Architecture and high-level signatures.)
At this point, LLM providers have more reliable LLM tool invocation.
- And API parameters that postulate (or force) tool invocation behavior.
- Still, not 100% reliable or expected.
In principle, LLM function calling can be replaced by using LLM graphs, [AA5].
- Though, at this point llm-graph provides computation over acyclic graphs only.
- On the other hand, llm-synthesize and llm-synthesize-with-tools use loops for multiple iterations over the tool invocation.
  - Again, the tool is external to the LLM. Tools are (most likely) running on “local” computers.
In Raku, LLM tooling specs can be (nicely) derived by introspection.
- So, package developers are encouraged to use declarator blocks as much as possible.
- Very often, though, it is easier to write an adapter function with specific (or simplified) input parameters.
  - See the last section “Adding plot tools”.
The package “LLM::Functions” provides a system of classes and subs that facilitate LLM function calling, [AA3].
- See the namespace LLM::Tooling:
  - Classes: LLM::Tool, LLM::ToolRequest, LLM::ToolResponse.
  - Subs: sub-info, llm-tool-definition, generate-llm-tool-response, llm-tool-request.
- A new LLM tool for the sub &f can be easily created with LLM::Tool.new(&f).
  - LLM::Tool uses llm-tool-definition which, in turn, uses sub-info.

Outline

Here is an outline of the exposition below:

Setup
Computation environment setup
Chemistry computations examples
Stoichiometry computations demonstrations
Define package functions as tools
Show how to define LLM-tools
Stoichiometry by LLM
Invoking LLM requests with LLM tools
“Thoughtful” response
Elaborated LLM answer based in LLM tools results
Adding plot tools
Enhancing the LLM answers with D3.js plots

Setup

Load packages:

use JSON::Fast;
use LLM::Functions;
use LLM::Tooling;
use Chemistry::Stoichiometry;
use JavaScript::D3;

Define LLM access configurations:

sink my $conf41-mini = llm-configuration('ChatGPT', model => 'gpt-4.1-mini', :8192max-tokens, temperature => 0.4);
sink my $conf-gemini-flash = llm-configuration('Gemini', model => 'gemini-2.0-flash', :8192max-tokens, temperature => 0.4);

JavaScript::D3

#%javascript
require.config({
     paths: {
     d3: 'https://d3js.org/d3.v7.min'
}});

require(['d3'], function(d3) {
     console.log(d3);
});

Chemistry computations examples

The package “Chemistry::Stoichiometry”, [AAp4], provides element data, a grammar (or parser) for chemical formulas, and subs for computing molecular masses and balancing equations. Here is an example of calling molecular-mass:

molecular-mass("SO2")

# 64.058

Balance chemical equation:

'Al + O2 -> Al2O3'
==> balance-chemical-equation

# [4*Al + 3*O2 -> 2*Al2O3]

Define package functions as tools

Define a few tools based in chemistry computations subs:

sink my @tools =
        LLM::Tool.new(&molecular-mass),
        LLM::Tool.new(&balance-chemical-equation)
        ;

Undefined type of parameter ⎡$spec⎦; continue assuming it is a string.

Make an LLM configuration with the LLM-tools:

sink my $conf = llm-configuration($conf41-mini, :@tools);

Remark: When llm-synthesize is given LLM configurations with LLM tools, it hands over the process to llm-synthesize-with-tools. This function then begins the LLM-tool interaction loop.

Stoichiometry by LLM

Here is a prompt requesting to compute molecular masses and to balance a certain chemical equation:

sink my $input = "What are the masses of SO2, O3, and C2H5OH? Also balance: C2H5OH + O2 = H2O + CO2."

The LLM invocation and result:

llm-synthesize(
        [$input, llm-prompt('NothingElse')('JSON')],
        e => $conf, 
        form => sub-parser('JSON'):drop)

# {balanced_equation => 1*C2H5OH + 3*O2 -> 2*CO2 + 3*H2O, masses => {C2H5OH => 46.069, O3 => 47.997, SO2 => 64.058}}

Remark: It order to see the LLM-tool interaction use the Boolean option (adverb) :echo of llm-synthesize.

“Thoughtful” response

Here is a very informative, “thoughtful” response for a quantitative Chemistry question:

#% markdown
my $input = "How many molecules a kilogram of water has? Use LaTeX for the formulas. (If any.)";

llm-synthesize($input, e => $conf)
==> { .subst(/'\[' | '\]'/, '$$', :g).subst(/'\(' | '\)'/, '$', :g) }() # Make sure LaTeX code has proper fences

Adding plot tools

It would be interesting (or fancy) to add a plotting tool. We can use text-list-plot of “Text::Plot”, [AAp5], or js-d3-list-plot of “JavaScript::D3”, [AAp6]. For both, the automatically derived tool specs — via the sub llm-tool-definition used by LLM::Tool — are somewhat incomplete. Here is the auto-result for js-d3-list-plot:

#llm-tool-definition(&text-list-plot)
llm-tool-definition(&js-d3-list-plot)

{
  "function": {
    "strict": true,
    "parameters": {
      "additionalProperties": false,
      "required": [
        "$data",
        ""
      ],
      "type": "object",
      "properties": {
        "$data": {
          "description": "",
          "type": "string"
        },
        "": {
          "description": "",
          "type": "string"
        }
      }
    },
    "type": "function",
    "name": "js-d3-list-plot",
    "description": "Makes a list plot (scatter plot) for a list of numbers or a list of x-y coordinates."
  },
  "type": "function"
}

The automatic tool-spec for js-d3-list-plot can be replaced with this spec:

my $spec = q:to/END/;
{
  "type": "function",
  "function": {
    "name": "jd-d3-list-plot",
    "description": "Creates D3.js code for a list-plot of the given arguments.",
    "parameters": {
      "type": "object",
      "properties": {
        "$x": {
          "type": "array",
          "description": "A list of a list of x-coordinates or x-labels",
          "items": {
            "anyOf": [
              { "type": "string" },
              { "type": "number" }
            ]
          }
        }
        "$y": {
          "type": "array",
          "description": "A list of y-coordinates",
          "items": {
            "type": "number"
          }
        }
      },
      "required": ["$x", "$y"]
    }
  }
}
END

my $t = LLM::Tool.new(&text-list-plot);
$t.json-spec = $spec;

Though, it is easier and more robust to define a new function that delegates to js-d3-list-plot — or other plotting function — and does some additional input processing that anticipates LLM derived argument values:

#| Make a string that represents a list-plot of the given arguments.
my sub data-plot(
    Str:D $x,             #= A list of comma separated x-coordinates or x-labels
    Str:D $y,             #= A list of comma separated y-coordinates
    Str:D :$x-label = '', #= Label of the x-axis
    Str:D :$y-label = '', #= Label of the y-axis
    Str:D :$title = '',   #= Plot title
    ) {
  
    my @x = $x.split(/<[\[\],"]>/, :skip-empty)».trim.grep(*.chars);
    my @y = $y.split(/<[\[\],"]>/, :skip-empty)».trim».Num;
      
    my @points = (@x Z @y).map({ %( variable => $_.head, value => $_.tail ) });
    js-d3-bar-chart(@points, :$x-label, :$y-label, title-color => 'Gray', background => '#1F1F1F', :grid-lines)
}

Here we add the new tool to the tool list above:

sink my @tool-objects =
        LLM::Tool.new(&molecular-mass),
        LLM::Tool.new(&balance-chemical-equation),
        LLM::Tool.new(&data-plot);

Here we make an LLM request for chemical molecules masses calculation and corresponding plotting — note that require to obtain a dictionary of the masses and plot:

my $input = q:to/END/;
What are the masses of SO2, O3, Mg2, and C2H5OH? 
Make a plot the obtained quantities: x-axes for the molecules, y-axis for the masses.
The plot has to have appropriate title and axes labels.
Return a JSON dictionary with keys "masses" and "plot".
END

# LLM configuration with tools
my $conf = llm-configuration($conf41-mini, tools => @tool-objects);

# LLM invocation
my $res = llm-synthesize([
        $input, 
        llm-prompt('NothingElse')('JSON')
    ], 
    e => $conf,
    form => sub-parser('JSON'):drop
);

# Type/structure of the result
deduce-type($res)

# Struct([masses, plot], [Hash, Str])

Here are result’s molecule masses:

$res<masses>

# {C2H5OH => 46.069, Mg2 => 48.61, O3 => 47.997, SO2 => 64.058}

Here is the corresponding plot:

#%js
$res<plot>

References

Articles, blog posts

[AA1] Anton Antonov, “LLM function calling workflows (Part 1, OpenAI)”, (2025), RakuForPrediction at WordPress.

[AA2] Anton Antonov, “LLM function calling workflows (Part 2, Google’s Gemini)”, (2025), RakuForPrediction at WordPress.

[AA3] Anton Antonov, “LLM function calling workflows (Part 3, Facilitation)”, (2025), RakuForPrediction at WordPress.

[AA4] Anton Antonov, “LLM function calling workflows (Part 4, Universal specs)”, (2025), RakuForPrediction at WordPress.

[AA5] Anton Antonov, “LLM::Graph”, (2025), RakuForPrediction at WordPress.

[Gem1] Google Gemini, “Gemini Developer API”.

[OAI1] Open AI, “Function calling guide”.

[WRI1] Wolfram Research, Inc., “LLM-Related Functionality” guide.

Packages

[AAp1] Anton Antonov, LLM::Functions, Raku package, (2023-2025), GitHub/antononcube.

[AAp2] Anton Antonov, WWW::OpenAI, Raku package, (2023-2025), GitHub/antononcube.

[AAp3] Anton Antonov, WWW::Gemini, Raku package, (2023-2025), GitHub/antononcube.

[AAp4] Anton Antonov, Chemistry::Stoichiometry, Raku package, (2021-2025), GitHub/antononcube.

[AAp5] Anton Antonov, Text::Plot, Raku package, (2022-2025), GitHub/antononcube.

[AAp6] Anton Antonov, JavaScript::D3, Raku package, (2022-2025), GitHub/antononcube.

LLM function calling workflows (Part 3, Facilitation)

Introduction

This document (notebook) shows how to efficiently do streamlined Function Calling workflows with Large Language Models (LLMs) of Gemini.

The Raku package “WWW::Gemini”, [AAp2], is used.

Examples and big picture

The rest of the document gives concrete code how to do streamline multiple-tool function calling with Gemini’s LLMs using Raku. Gemini’s function calling example “Parallel Function Calling”, [Gem1], is followed.

This document belongs to a collection of documents describing how to do LLM function calling with Raku.

Compared to the previously described LLM workflows with OpenAI, [AA1], and Gemini, [AA2], the Gemini LLM workflow in this document demonstrates:

Use of multiple tools (parallel function calling)
Automatic generation of hashmap (or JSON) tool descriptors
Streamlined computation of multiple tool results from multiple LLM requests

The streamlining is achieved by using the provided by “LLM::Functions”, [AAp3]:

Classes LLM::Tool, LLM::ToolRequest, and LLM::ToolResult
Subs llm-tool-definition and generate-llm-tool-result
- The former sub leverages Raku’s introspection features.
- The latter sub matches tools and requests in order to compute tool responses.

Setup

Load packages:

use JSON::Fast;
use Data::Reshapers;
use Data::TypeSystem;
use LLM::Tooling;
use WWW::Gemini;

Choose a model:

my $model = "gemini-2.0-flash";

Workflow

Define a local function

Define a few subs — tools — with sub- and argument descriptions (i.e. attached Pod values, or declarator blocks):

#| Powers the spinning disco ball.
sub power-disco-ball-impl(
    Int:D $power #= Whether to turn the disco ball on or off.
    ) returns Hash {
    return { status => "Disco ball powered " ~ ($power ?? 'on' !! 'off') };
}
#= A status dictionary indicating the current state.

#| Play some music matching the specified parameters.
sub start-music-impl(
    Int:D $energetic, #=  Whether the music is energetic or not.
    Int:D $loud       #= Whether the music is loud or not.
    ) returns Hash {
    my $music-type = $energetic ?? 'energetic' !! 'chill';
    my $volume = $loud ?? 'loud' !! 'quiet';
    return { music_type => $music-type, volume => $volume };
    #= A dictionary containing the music settings.
}

#| Dim the lights.
sub dim-lights-impl(
    Numeric:D $brightness #= The brightness of the lights, 0.0 is off, 1.0 is full.
    ) returns Hash {
    return { brightness => $brightness };
}
#= A dictionary containing the new brightness setting.

Remark: See the corresponding Python definitions in the section “Parallel Function Calling” of [Gem1].

The sub llm-tool-definition can be used to automatically generate the Raku-hashmaps or JSON-strings of the tool descriptors in the (somewhat universal) format required by LLMs:

llm-tool-definition(&dim-lights-impl, format => 'json')

# {
#   "function": {
#     "type": "function",
#     "name": "dim-lights-impl",
#     "strict": true,
#     "description": "Dim the lights.",
#     "parameters": {
#       "required": [
#         "$brightness"
#       ],
#       "additionalProperties": false,
#       "type": "object",
#       "properties": {
#         "$brightness": {
#           "type": "number",
#           "description": "The brightness of the lights, 0.0 is off, 1.0 is full."
#         }
#       }
#     }
#   },
#   "type": "function"
# }

Remark: The sub llm-tool-description is invoked in LLM::Tool.new. Hence (ideally) llm-tool-description would not be user-invoked that often.

These are the tool descriptions to be communicated to Gemini:

my @tools =
{
    :name("power-disco-ball-impl"), 
    :description("Powers the spinning disco ball."), 
    :parameters(
        {
            :type("object")
            :properties( {"\$power" => {:description("Whether to turn the disco ball on or off."), :type("integer")}}), 
            :required(["\$power"]), 
        }), 
},
{
    :name("start-music-impl"), 
    :description("Play some music matching the specified parameters."), 
    :parameters(
        {
            :type("object")
            :properties({
                "\$energetic" => {:description("Whether the music is energetic or not."), :type("integer")}, 
                "\$loud" => {:description("Whether the music is loud or not."), :type("integer")}
            }), 
            :required(["\$energetic", "\$loud"]), 
        }),
},
{
    :name("dim-lights-impl"), 
    :description("Dim the lights."), 
    :parameters(
        {
            :type("object")
            :properties({"\$brightness" => {:description("The brightness of the lights, 0.0 is off, 1.0 is full."), :type("number")}}), 
            :required(["\$brightness"]), 
        }), 
};

deduce-type(@tools)

# Vector(Struct([description, name, parameters], [Str, Str, Hash]), 3)

Here are additional tool-mode configurations (see “Function calling modes” of [Gem1]):

my %toolConfig =
  functionCallingConfig => {
    mode => "ANY",
    allowedFunctionNames => <power-disco-ball-impl start-music-impl dim-lights-impl>
  };

# {functionCallingConfig => {allowedFunctionNames => (power-disco-ball-impl start-music-impl dim-lights-impl), mode => ANY}}

First communication with Gemini

Initialize messages:

# User prompt
my $prompt = 'Turn this place into a party!';

# Prepare the API request payload
my @messages = [{role => 'user',parts => [ %( text => $prompt ) ]}, ];

# [{parts => [text => Turn this place into a party!], role => user}]

Send the first chat completion request:

my $response = gemini-generate-content(
    @messages,
    :$model,
    :@tools,
    :%toolConfig
);

deduce-type($response)

# Struct([candidates, modelVersion, responseId, usageMetadata], [Hash, Str, Str, Hash])

deduce-type($response)

# Struct([candidates, modelVersion, responseId, usageMetadata], [Hash, Str, Str, Hash])

The response is already parsed from JSON to Raku. Here is its JSON form:

to-json($response)

# {
#   "candidates": [
#     {
#       "avgLogprobs": -0.0012976408004760742,
#       "content": {
#         "parts": [
#           {
#             "functionCall": {
#               "name": "start-music-impl",
#               "args": {
#                 "$energetic": 1,
#                 "$loud": 1
#               }
#             }
#           },
#           {
#             "functionCall": {
#               "name": "power-disco-ball-impl",
#               "args": {
#                 "$power": 1
#               }
#             }
#           },
#           {
#             "functionCall": {
#               "args": {
#                 "$brightness": 0.5
#               },
#               "name": "dim-lights-impl"
#             }
#           }
#         ],
#         "role": "model"
#       },
#       "safetyRatings": [
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_HATE_SPEECH"
#         },
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_DANGEROUS_CONTENT"
#         },
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_HARASSMENT"
#         },
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT"
#         }
#       ],
#       "finishReason": "STOP"
#     }
#   ],
#   "usageMetadata": {
#     "candidatesTokensDetails": [
#       {
#         "tokenCount": 30,
#         "modality": "TEXT"
#       }
#     ],
#     "promptTokensDetails": [
#       {
#         "tokenCount": 113,
#         "modality": "TEXT"
#       }
#     ],
#     "promptTokenCount": 113,
#     "candidatesTokenCount": 30,
#     "totalTokenCount": 143
#   },
#   "responseId": "sOxFaOrFF-SfnvgPgITLqQ8",
#   "modelVersion": "gemini-2.0-flash"
# }

Refine the response with functional calls

The following copy of the messages is not required, but it makes repeated experiments easier:

my @messages2 = @messages;

# [{parts => [text => Turn this place into a party!], role => user}]

Let us define an LLM::Tool object for each tool:

my @toolObjects = [&power-disco-ball-impl, &start-music-impl, &dim-lights-impl].map({ LLM::Tool.new($_) });

.say for @toolObjects

# LLMTool(power-disco-ball-impl, Powers the spinning disco ball.)
# LLMTool(start-music-impl, Play some music matching the specified parameters.)
# LLMTool(dim-lights-impl, Dim the lights.)

Make an LLM::Request object for each request from the (first) LLM response:

my @requestObjects = $response<candidates>»<content>»<parts>.&flatten»<functionCall>.map({ LLM::ToolRequest.new( $_<name>, $_<args>) });

.say for @requestObjects

# LLMToolRequest(start-music-impl, :$loud(1), :$energetic(1), :id(Whatever))
# LLMToolRequest(power-disco-ball-impl, :$power(1), :id(Whatever))
# LLMToolRequest(dim-lights-impl, :$brightness(0.5), :id(Whatever))

Using the relevant tool for each request compute tool’s response (which are LLM::ToolResponse objects):

.say for @requestObjects.map({ generate-llm-tool-response(@toolObjects, $_) })».output

# {music_type => energetic, volume => loud}
# {status => Disco ball powered on}
# {brightness => 0.5}

Alternatively, the LLM::ToolResponse objects can be converted into hashmaps structured according a particular LLM function calling style (Gemini in this case):

.say for @requestObjects.map({ generate-llm-tool-response(@toolObjects, $_) })».Hash('Gemini')

# {functionResponse => {name => start-music-impl, response => {content => {music_type => energetic, volume => loud}}}}
# {functionResponse => {name => power-disco-ball-impl, response => {content => {status => Disco ball powered on}}}}
# {functionResponse => {name => dim-lights-impl, response => {content => {brightness => 0.5}}}}

Process the response:

Make a request object for each function call request
Compute the tool results
Form corresponding user message with those results
Send the messages to the LLM

my $assistant-message = $response<candidates>[0]<content>;
if $assistant-message<parts> {

    # Find function call parts and make corresponding tool objects
    my @requestObjects;
    for |$assistant-message<parts> -> %part {
        if %part<functionCall> {
            @requestObjects.push: LLM::ToolRequest.new( %part<functionCall><name>, %part<functionCall><args> ) 
        }
    }    

    # Add assistance message
    @messages2.push($assistant-message);

    # Compute tool responses
    my @funcParts = @requestObjects.map({ generate-llm-tool-response(@toolObjects, $_) })».Hash('Gemini');

    # Make and add the user response
    my %function-response =
        role => 'user',
        parts => @funcParts;

    @messages2.push(%function-response);
                
    # Send the second request with function result
    my $final-response = gemini-generate-content(
        @messages2,
        :@tools,
        :$model,
        format => "raku"
    );
                
    say "Assistant: ", $final-response<candidates>[0]<content><parts>».<text>.join("\n");

} else {
    say "Assistant: $assistant-message<content>";
}

# Assistant: Alright! I've started some energetic and loud music, turned on the disco ball, and dimmed the lights to 50% brightness. Let's get this party started!

Remark Compared to the workflows in [AA1, AA2] the code above in simpler, more universal and robust, and handles all tool requests

Conclusion

We can observe and conclude that LLM function calling workflows are greatly simplified by:

Leveraging Raku introspection
- This requires documenting the subs and their parameters.
Using dedicated classes that represent tool:
- Definitions, (LLM::Tool)
- Requests, (LLM::ToolRequest)
- Responses, (LLM::ToolResponse)
Having a sub (generate-llm-tool-response) that automatically matches request objects to tool objects and produces the corresponding response objects.
- Note the Gemini’s documentation does not show that matching in the corresponding function calling example “Parallel Function Calling”.

Raku’s LLM tools automation is similar to Gemini’s “Automatic Function Calling (Python Only)”.

The Wolfram Language LLM tooling functionalities are reflected in Raku’s “LLM::Tooling”, [WRI1].

References

Articles, blog posts

[AA1] Anton Antonov, “LLM function calling workflows (Part 1, OpenAI)”, (2025), RakuForPrediction at WordPress.

[AA2] Anton Antonov, “LLM function calling workflows (Part 2, Google’s Gemini)”, (2025), RakuForPrediction at WordPress.

[AA3] Anton Antonov, “LLM function calling workflows (Part 3, Facilitation)”, (2025), RakuForPrediction at WordPress.

[Gem1] Google Gemini, “Gemini Developer API”.

[WRI1] Wolfram Research, Inc. “LLM-Related Functionality” guide.

Packages

[AAp1] Anton Antonov, WWW::OpenAI Raku package, (2023-2025), GitHub/antononcube.

[AAp2] Anton Antonov, WWW::Gemini Raku package, (2023-2025), GitHub/antononcube.

[AAp3] Anton Antonov, LLM::Functions Raku package, (2023-2025), GitHub/antononcube.

LLM function calling workflows (Part 2, Google’s Gemini)

Introduction

This document (notebook) shows how to do Function Calling workflows with Large Language Models (LLMs) of Google’s Gemini.

The Raku package “WWW::Gemini”, [AAp2], is used.

Examples and big picture

The rest of the document gives concrete code how to do function calling with Gemini’s LLMs using Raku.

There are similar workflows, [AA1], with other LLM providers. (Like, OpenAI.) They follow the same structure, although there are some small differences. (Say, in the actual specifications of tools.)

This document belongs to a collection of documents describing how to do LLM function calling with Raku.

The Gemini LLM workflow in this document is quite similar to the OpenIA workflow described in [AA1]. While there are variations in the tool configurations and how the elements of the LLM responses are obtained, the overall procedure outline and diagrams in [AA1] also apply to the workflows presented here.

Setup

Load packages:

use WWW::Gemini;
use JSON::Fast;

Choose a model:

my $model = "gemini-2.0-flash";

Workflow

Define a local function

This is the “tool” to be communicated to Gemini. (I.e. define the local function/sub.)

sub get-current-weather(Str:D $location, Str:D $unit = "fahrenheit") returns Str {
    return "It is currently sunny in $location with a temperature of 72 degrees $unit.";
}

Define the function specification (as prescribed in Gemini’s function calling documentation):

my %weather-function = %(
    name => 'get-current-weather',
    description => 'Get the current weather in a given location',
    parameters => %(
        type => 'object',
        properties => %(
            location => %(
                type => 'string',
                description => 'The city and state, e.g., Boston, MA'
            )
        ),
        required => ['location']
    )
);

First communication with Gemini

Initialize messages and tools:

# User prompt
my $prompt = 'What is the weather like in Boston, MA, USA?';

# Prepare the API request payload
my @messages = [{role => 'user',parts => [ %( text => $prompt ) ]}, ];

my @tools = [%weather-function, ];

# [{description => Get the current weather in a given location, name => get-current-weather, parameters => {properties => {location => {description => The city and state, e.g., Boston, MA, type => string}}, required => [location], type => object}}]

Send the first chat completion request:

my $response = gemini-generate-content(
    @messages,
    :$model,
    :@tools
);

The response is already parsed from JSON to Raku. Here is its JSON form:

to-json($response)

# {
#   "usageMetadata": {
#     "totalTokenCount": 50,
#     "promptTokensDetails": [
#       {
#         "tokenCount": 41,
#         "modality": "TEXT"
#       }
#     ],
#     "candidatesTokenCount": 9,
#     "candidatesTokensDetails": [
#       {
#         "tokenCount": 9,
#         "modality": "TEXT"
#       }
#     ],
#     "promptTokenCount": 41
#   },
#   "modelVersion": "gemini-2.0-flash",
#   "candidates": [
#     {
#       "finishReason": "STOP",
#       "safetyRatings": [
#         {
#           "category": "HARM_CATEGORY_HATE_SPEECH",
#           "probability": "NEGLIGIBLE"
#         },
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_DANGEROUS_CONTENT"
#         },
#         {
#           "probability": "NEGLIGIBLE",
#           "category": "HARM_CATEGORY_HARASSMENT"
#         },
#         {
#           "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
#           "probability": "NEGLIGIBLE"
#         }
#       ],
#       "content": {
#         "parts": [
#           {
#             "functionCall": {
#               "args": {
#                 "location": "Boston, MA"
#               },
#               "name": "get-current-weather"
#             }
#           }
#         ],
#         "role": "model"
#       },
#       "avgLogprobs": -3.7914659414026473e-06
#     }
#   ],
#   "responseId": "zDpEaIClFpu97dcPpqOWiA8"
# }

Refine the response with functional calls

The following copy of the messages is not required, but it makes repeated experiments easier:

my @messages2 = @messages;

# [{parts => [text => What is the weather like in Boston, MA, USA?], role => user}]

Process the response — invoke the tool, give the tool result to the LLM, get the LLM answer:

my $assistant-message = $response<candidates>[0]<content>;
if $assistant-message<parts> {

    for |$assistant-message<parts> -> %part {
        if %part<functionCall> {
            
            @messages2.push($assistant-message);

            my $func-name = %part<functionCall><name>;
            my %args = %part<functionCall><args>;

            
            if $func-name eq 'get-current-weather' {
                my $location = %args<location>;
                my $weather = get-current-weather($location);

                my %function-response =
                            role => 'user',
                            parts => [{ 
                                functionResponse => {
                                    name => 'get-current-weather',
                                    response => %( content => $weather )
                                } 
                            }];

                @messages2.push(%function-response);
                
                # Send the second request with function result
                my $final-response = gemini-generate-content(
                    @messages2,
                    :@tools,
                    :$model,
                    format => "raku"
                );
                
                say "Assistant: ", $final-response<candidates>[0]<content><parts>».<text>.join("\n");

                last
            }
        }
    }
} else {
    say "Assistant: $assistant-message<content>";
}

# Assistant: The weather in Boston, MA is currently sunny with a temperature of 72 degrees Fahrenheit.

Remark: Note that if get-current-weather is applied then the loop above immediately finishes.

References

Articles, blog posts

[AA1] Anton Antonov, “LLM function calling workflows (Part 1, OpenAI)”, (2025), RakuForPrediction at WordPress.

[AA2] Anton Antonov,
“LLM function calling workflows (Part 2, Google’s Gemini)”, (2025), RakuForPrediction at WordPress.

Packages

[AAp1] Anton Antonov, WWW::OpenAI Raku package, (2023-2025), GitHub/antononcube.

[AAp2] Anton Antonov, WWW::Gemini Raku package, (2023-2025), GitHub/antononcube.

[AAp3] Anton Antonov, LLM::Functions Raku package, (2023-2025), GitHub/antononcube.

LLM function calling workflows (Part 1, OpenAI)

Introduction

This document (notebook) shows how to do Function Calling workflows with Large Language Models (LLMs) of OpenAI.

The Raku package “WWW::OpenAI”, [AAp1], is used.

Outline of the overall process

The overall process is (supposed to be) simple:

Implement a “tool”, i.e. a function/sub
- The tool is capable of performing (say, quickly and reliably) certain tasks.
- More than one tool can be specified.
Describe the tool(s) using a certain JSON format
- The JSON description is to be “understood” by the LLM.
- JSON-schema is used for the arguments.
- Using the description, the LLM figures out when to make requests for computations with the tool and with what parameters and corresponding values.
Make a first call to the LLM using suitably composed messages that have the tool JSON description(s).
Examine the response of the LLM:
If the response indicates that the (local) tool has to be evaluated:
- Process the tool names and corresponding parameters.
- Make a new message with the tool result(s).
- Send the messages to the LLM.
- Goto Step 4.
Otherwise, give that “final” response.

(Currently) OpenAI indicates its tool evaluation requests with having the rule finish_reason => tool_calls in its responses.

Diagram

Here is a Mermaid-JS diagram that shows single-pass LLM-and-tool interaction:

Remark: Instead of a loop — as in the outline above — only one invocation of a local tool is shown in the diagram.

Examples and big picture

The rest of the document gives concrete code how to do function calling with OpenAI’s LLMs using Raku.

There are similar workflows with other LLM providers. (Like, Google’s Gemini.) They follow the same structure, although there are some small differences. (Say, in the actual specifications of tools.)

It would be nice to have:

Universal programming interface for those function calling interfaces.
Facilitation of tool descriptions derivations.
- Via Raku’s introspection or using suitable LLM prompts.
  - “LLM::Functions”, [AAp3], can be used for both approaches.

This document belongs to a collection of documents describing how to do LLM function calling with Raku.

Setup

Load packages:

use WWW::OpenAI;
use JSON::Fast;

Choose a model:

my $model = "gpt-4.1";

Workflow

Define a local function

This is the “tool” to be communicated to OpenAI. (I.e. define the local function/sub.)

sub get-current-weather(Str $location, Str $unit = "fahrenheit") returns Str {
    return "It is currently sunny in $location with a temperature of 72 degrees $unit.";
}

Define the function specification (as prescribed in OpenAI’s function calling documentation):

my $function-spec = {
    type => "function",
    function => {
        name => "get-current-weather",
        description => "Get the current weather for a given location",
        parameters => {
            type => "object",
            properties => {
                '$location' => {
                    type => "string",
                    description => "The city and state, e.g., San Francisco, CA"
                },
                '$unit' => {
                    type => "string",
                    enum => ["celsius", "fahrenheit"],
                    description => "The temperature unit to use"
                }
            },
            required => ["location"]
        }
    }
};

First communication with OpenAI

Initialize messages and tools:

my @messages =
    {role => "system", content =>  "You are a helpful assistant that can provide weather information."},
    {role => "user", content => "What's the weather in Boston, MA?"}
    ;

my @tools = [$function-spec,];

Send the first chat completion request:

my $response = openai-chat-completion(
    @messages,
    :@tools,
    :$model,
    max-tokens => 4096,
    format => "raku",
    temperature => 0.45
);

# [{finish_reason => tool_calls, index => 0, logprobs => (Any), message => {annotations => [], content => (Any), refusal => (Any), role => assistant, tool_calls => [{function => {arguments => {"$location":"Boston, MA"}, name => get-current-weather}, id => call_ROi3n0iICSrGbetBKZ9KVG4E, type => function}]}}]

Refine the response with functional calls

The following copy of the messages is not required, but it makes repeated experiments easier:

my @messages2 = @messages;

Process the response — invoke the tool, give the tool result to the LLM, get the LLM answer:

my $assistant-message = $response[0]<message>;
if $assistant-message<tool_calls> {

    @messages2.push: {
        role => "assistant",
        tool_calls => $assistant-message<tool_calls>
    };

    my $tool-call = $assistant-message<tool_calls>[0];
    my $function-name = $tool-call<function><name>;
    my $function-args = from-json($tool-call<function><arguments>);
    
    if $function-name eq "get-current-weather" {
        my $result = get-current-weather(
            $function-args{'$location'} // $function-args<location>,
            $function-args{'$unit'} // $function-args<unit> // "fahrenheit"
        );
        @messages2.push: {
            role => "tool",
            content => $result,
            tool_call_id => $tool-call<id>
        };
        
        # Send the second request with function result
        my $final-response = openai-chat-completion(
            @messages2,
            :@tools,
            #tool_choice => "auto",
            :$model,
            format => "raku"
        );
        say "Assistant: $final-response[0]<message><content>";
    }
} else {
    say "Assistant: $assistant-message<content>";
}

# Assistant: The weather in Boston, MA is currently sunny with a temperature of 72

Show all messages:

.say for @messages2

# {content => You are a helpful assistant that can provide weather information., role => system}
# {content => What's the weather in Boston, MA?, role => user}
# {role => assistant, tool_calls => [{function => {arguments => {"$location":"Boston, MA"}, name => get-current-weather}, id => call_ROi3n0iICSrGbetBKZ9KVG4E, type => function}]}
# {content => It is currently sunny in Boston, MA with a temperature of 72 degrees fahrenheit., role => tool, tool_call_id => call_ROi3n0iICSrGbetBKZ9KVG4E}

In general, there should be an evaluation loop that checks the finishing reason(s) in the LLM answers and invokes the tools as many times as it is required. (I.e., there might be several back-and-forth exchanges in the LLM, requiring different tools or different tool parameters.)

References

[AAp1] Anton Antonov, WWW::OpenAI Raku package, (2023-2025), GitHub/antononcube.

[AAp2] Anton Antonov, WWW::Gemini Raku package, (2023-2025), GitHub/antononcube.

[AAp3] Anton Antonov, LLM::Functions Raku package, (2023-2025), GitHub/antononcube.

Menu

Introduction

Big picture

The whole series

Overall comments and observations

Outline

Setup

JavaScript::D3

Chemistry computations examples

Define package functions as tools

Stoichiometry by LLM

“Thoughtful” response

Adding plot tools

References

Articles, blog posts

Packages

Introduction

Examples and big picture

Setup

Workflow

Define a local function

First communication with Gemini

Refine the response with functional calls

Conclusion

References

Articles, blog posts

Packages

Introduction

Examples and big picture

Setup

Workflow

Define a local function

First communication with Gemini

Refine the response with functional calls

References

Articles, blog posts

Packages

Introduction

Outline of the overall process

Diagram

Examples and big picture

Setup

Workflow

Define a local function

First communication with OpenAI

Refine the response with functional calls

References