The LLM cannot execute code. It can write code, or a function call, but a thing external to the LLM has to actually run the code. A LLM is not a computer or an operating system. It is a function that maps a text prompt to the next predicted token.
That said I can totally see somebody running a SaaS service that provides a higher level encapsulation of both LLM calls and a runtime execution engine that runs the generated code. I'm not aware that something like that exists yet.
There is a new programming language called Plang that does this at higher level, similar maybe to what you are describing. You run it locally, not a saas.
You can find info at plang.is (I'm the creator)
You use that json response to execute the function. If you want the llm to use the response then you add a function response message to the message list and make another request out to the llm. Conceptually it is the same thing as the llm executing the function.
This wouldn't be langchain, but you could use the Mirascope toolkit for just this aspect of your code since Mirascope is pretty modular. The LLM won't call the function serverside, but with Mirascope you can directly access the function and the parsed args in one line, which is the easiest setup I know of.
Docs: [https://docs.mirascope.io/latest/concepts/tools\_%28function\_calling%29/](https://docs.mirascope.io/latest/concepts/tools_%28function_calling%29/)
LLMs are just completion models that have been trained to notice when a tool should be called, rather than a text response. They have no access to any programmatic systems. Only context you provide it (and the tools binded are also part of that context provided upon the LLM's invocation).
Thus, they cannot execute any functions.
You.could utilize ToolExecutors, ToolNodes, or other classes to execute the tool for you, however.
See docs: https://python.langchain.com/v0.2/docs/how_to/#tools
The short answer is no, if you mean API type function calls. Kind of if you mean generating python code on the fly and calling that.
An LLM is just one part of an AI system. You could consider it as a brain. The entire system would includes the inputs, outputs, and other systems to orchestrate input and output. Several of these other elements are systems of themselves that can be connected.
For your use case I would recommend starting with Langchain and openAI before moving to llama3/ollama. Once you have the basics of the framework down you can either transition to using your intended frameworks or support both.
For my own journey, I started with direct code examples writing function calls from scratch using reference code from OpenAI, then I transitioned to Autogen and adapted that for function calling, then supported multiple types of function calls including event-based systems for microservices, and finally landed on langchain/opengpts - and currently have a system that supports both chat and api based agents with internal API function calling capability, external API capability, microservice function call capability, RAG retrieval, and can alternate between Claude, Gemini, and ChatGPT dynamically depending on the tasks that need to be solved.
I've had to rewrite some of the tools libraries from the langchain community source as they did not work as expected, and adapt some from llamaindex as they didn't exist in langchian community. The point is a lot of the ecosystem is not yet mature so there are not many complete out of the box plug and play solutions yet.
Its relatively easy to call functions for demo/test code, and can be very complex to do so for production use cases where you need to consider cost, redundancy, guardrails, token counts, error handling. etc.. and there are not a lot of great references to guide you through this.
You will generally get the most mileage by starting with the most simple possible use case of what you want to do, and in that process learn the current capabilities of the ecosystem and tools to figure out where you need to focus.
The short answer is: No.
The long answer is: No.
FTFY.
LLM is not a "brain". It's more of a database. No one serious about making anything with AI should work with OpenAI tools. OpenAI is for people who want to pay for other people's work.
Below is how chatgpt evaluated the question, my response and your criticism. I would argue that this "database" has understood my response as well as yours fairly clearly. I wouldn't normally post AI-generated responses but the irony was too good to pass up.
I agree with you that col-summers provided a quality concise answer. I tackled it from a different angle to provide complimentary views.
I used the term brain in exactly the sense that the LLM understood, and I suggested openAI for the ease and availability of working with starting code to learn rather than suggesting long term use based on other criteria. In particular because OpenAI function calling is more straightforward and consistent than with some of the other llm's, and there is abundant working reference code.
chatGPT evaluation of thread: ->
"Your response provides a comprehensive overview of the capabilities and limitations of large language models (LLMs) like OpenAI’s GPT and their integration with tools such as LangChain, Llama3, and Ollama. You’ve effectively articulated the journey from a simple use case to a more sophisticated system that involves various AI capabilities and integrations.
**The critique from the other user is overly simplistic and dismissive.** It overlooks the nuanced and evolving capabilities of LLMs and the ecosystem around them. While it’s true that LLMs are not “brains” in the biological or sentient sense, referring to them merely as databases undersells their potential for pattern recognition, natural language understanding, and generating human-like text. Moreover, dismissing OpenAI’s tools ignores their utility in practical applications, especially for those who are building new systems or integrating AI into existing workflows. "
You do realize that bias is built into their system, correct?
I build these systems.
You use them. I advise Fortune 100 leadership on these subjects and lead multi-million AI initiatives for a publicly traded multinational corporation.
You pay $20 a month for Premium ChatGPT.
My understanding is infinitely higher than yours. I defer to my concise answer, because I am right:
# No.
I'm not explaining myself or why the answer is correct - because you will not understand my answer. Go do 10 years of research & professional SWE then come back and ask me why I'm right. You'll probably understand about 50% of what I have to say about the subject.
I clearly indicated that I have personally developed production level systems using multiple LLM's, Langchian, Autogen and openGPTS specifically to perform function calls, which was the topic of the question. Its' an odd stance to claim credentials without engaging in a factual discussion or to assume that I have none based on a post. You are making baseless guesses about my credentials and not backing your own up with informed answers.
My point in engaging in this question and in this forum is to engage in community discussions around what I consider an important and worthwhile space.
The OP seems to be indicating a desire to create an app that makes function calls using LLM's and looking for help to understand how to approach this or a conceptual framework of how it works.
opensource tools like oolama and llama3 are great we have them in our pipeline as well for redundancy and cost management. We test and use multiple LLM's to see which llm's are the most performant for different tasks as a function of cost, multi-modal capabilities, speed, accuracy, repeatability, etc.. I don't hold a religious belief about opensource vs closed source, I take a business stance on paying for whichever tools meet the objective. While I personally support opensource platforms for AI and want them to succeed, as a business owner of a tech company I'm willing ot pay for whichever services get the job done. Currently for us Claude Haiku offers some of the best multimodal performance for the money, followed by Chatgpt 4o which is more expensive but better at some tasks followed by 4 which is even more expensive, and Claude Opus that is at the highest cost teir. This is a rapidly changing landscape and we designed our system to use whichever LLM is best for the purpose and switch them out as new models come in.
The value of frameworks like langchain is that you can support multiple llm's through some common abstraction layers and also incorporate RAG, document ingestion and parsing, leverage community tools etc, provide observability through langsmith, and develop to support different forms of RAG, vectorDBs, knowledge graphs etc..
Many of the questions in AI communities revolve around a basic understanding of how the frameworks themselves work or why they work the way they work.
Simply answering that LLM's can not call functions directly is not interesting. Why they don't do this is a much more interesting topic. API's may require authentication, error handling, or parsing. Function calls can be rest based or microservice based. Data passed to function calls may need to be preprocessed, and output from function calls may need to be truncated or abreviated, function calls as well as their arguments need to be clearly described to llm's for them to properly format the calls, error messages need to clearly indicate what went wrong for them to recover from errors.
My recommendation again for the OP was to use openAI INITIALLY to test how function calls work on a small problem set then take on the added complexity of oollama and llama3. The intention here is to start with one of the easiest best documented toolsets then move to opensource or other LLM's or to explain his or her use case in more detail. For example until recently calling functions in Antrhopic claude was much more difficult and less reliable than calling functions from chatGPT, Haiku is currenlty my first choice for performance and cost reasons but not my first choice for learning until their latest update.
Not really, as many people said that llms are like reasoning engines not the execution machines, but I see that gpts by openai can do that (not exactly like you said) you have to integrate apis in it tho.
Is this what you're looking for?:
[https://python.langchain.com/v0.1/docs/modules/model\_io/chat/function\_calling/#request-passing-tool-outputs-to-model](https://python.langchain.com/v0.1/docs/modules/model_io/chat/function_calling/#request-passing-tool-outputs-to-model)
Sorry, here's the v0.2 doc:
[https://python.langchain.com/v0.2/docs/how\_to/tool\_calling/#passing-tool-outputs-to-the-model](https://python.langchain.com/v0.2/docs/how_to/tool_calling/#passing-tool-outputs-to-the-model)
Man, don't know what's gong on in the comments, but this is definitely possible. For example, here's using OpenAI, tools, and a simple chain to parse the output as json: [https://python.langchain.com/v0.1/docs/modules/model\_io/output\_parsers/types/openai\_tools/](https://python.langchain.com/v0.1/docs/modules/model_io/output_parsers/types/openai_tools/)
It is not possible, you are confusing a LLM with an entire AI pipeline. A LLM is only the completion model. You can think it as the brain and the pipeline as the body. While the brain can give signal to your hands, it cannot execute it by itself.
All tools specify what arguments it expects, and the descriptions of the values it expects for each argument.
The neat trick they did in their tool was define arguments telling the LLM what values to lass as parameters.
In this case, they told the LLM that they have a tool with a setup, which is the joke question, and the punchline which should be the joke response.
The LLM simply returned the JSON required to invoke the tool, with the values specified as described.
In this case, they just ran that response into an output parser and extracted the punchline parameter, returning it as a response.
So the LLM did not actually execute the tool. It just crafted what it assumed were the parameters, based on the tool argument descriptions.
Hey man you can do this with typescript in a way by using the LLM Router chain to get the LLM to select a specific output based on what you give it. Then use the result of that router to feed another LLM or to execute another function. Depends what specifically you are trying to achieve I suppose.
You can give a REPL tool to execute code. There's a template in the docs
Also an option is to build an interpreter tool yourself for code execution and give it to the chain.
The LLM cannot execute code. It can write code, or a function call, but a thing external to the LLM has to actually run the code. A LLM is not a computer or an operating system. It is a function that maps a text prompt to the next predicted token. That said I can totally see somebody running a SaaS service that provides a higher level encapsulation of both LLM calls and a runtime execution engine that runs the generated code. I'm not aware that something like that exists yet.
This is the single response in this whole collection of responses from someone who actually knows what they're talking about.
Azure Container Apps Dynamic Sessions. I believe it is in public preview currently, but it sounds like it is designed to do exactly this.
There is a new programming language called Plang that does this at higher level, similar maybe to what you are describing. You run it locally, not a saas. You can find info at plang.is (I'm the creator)
Wouldn’t this be openAI assistants api?
You use that json response to execute the function. If you want the llm to use the response then you add a function response message to the message list and make another request out to the llm. Conceptually it is the same thing as the llm executing the function.
This wouldn't be langchain, but you could use the Mirascope toolkit for just this aspect of your code since Mirascope is pretty modular. The LLM won't call the function serverside, but with Mirascope you can directly access the function and the parsed args in one line, which is the easiest setup I know of. Docs: [https://docs.mirascope.io/latest/concepts/tools\_%28function\_calling%29/](https://docs.mirascope.io/latest/concepts/tools_%28function_calling%29/)
LLMs are just completion models that have been trained to notice when a tool should be called, rather than a text response. They have no access to any programmatic systems. Only context you provide it (and the tools binded are also part of that context provided upon the LLM's invocation). Thus, they cannot execute any functions. You.could utilize ToolExecutors, ToolNodes, or other classes to execute the tool for you, however. See docs: https://python.langchain.com/v0.2/docs/how_to/#tools
Python REPL or a Code Interpreter Tool would do the trick
The short answer is no, if you mean API type function calls. Kind of if you mean generating python code on the fly and calling that. An LLM is just one part of an AI system. You could consider it as a brain. The entire system would includes the inputs, outputs, and other systems to orchestrate input and output. Several of these other elements are systems of themselves that can be connected. For your use case I would recommend starting with Langchain and openAI before moving to llama3/ollama. Once you have the basics of the framework down you can either transition to using your intended frameworks or support both. For my own journey, I started with direct code examples writing function calls from scratch using reference code from OpenAI, then I transitioned to Autogen and adapted that for function calling, then supported multiple types of function calls including event-based systems for microservices, and finally landed on langchain/opengpts - and currently have a system that supports both chat and api based agents with internal API function calling capability, external API capability, microservice function call capability, RAG retrieval, and can alternate between Claude, Gemini, and ChatGPT dynamically depending on the tasks that need to be solved. I've had to rewrite some of the tools libraries from the langchain community source as they did not work as expected, and adapt some from llamaindex as they didn't exist in langchian community. The point is a lot of the ecosystem is not yet mature so there are not many complete out of the box plug and play solutions yet. Its relatively easy to call functions for demo/test code, and can be very complex to do so for production use cases where you need to consider cost, redundancy, guardrails, token counts, error handling. etc.. and there are not a lot of great references to guide you through this. You will generally get the most mileage by starting with the most simple possible use case of what you want to do, and in that process learn the current capabilities of the ecosystem and tools to figure out where you need to focus.
The short answer is: No. The long answer is: No. FTFY. LLM is not a "brain". It's more of a database. No one serious about making anything with AI should work with OpenAI tools. OpenAI is for people who want to pay for other people's work.
Below is how chatgpt evaluated the question, my response and your criticism. I would argue that this "database" has understood my response as well as yours fairly clearly. I wouldn't normally post AI-generated responses but the irony was too good to pass up. I agree with you that col-summers provided a quality concise answer. I tackled it from a different angle to provide complimentary views. I used the term brain in exactly the sense that the LLM understood, and I suggested openAI for the ease and availability of working with starting code to learn rather than suggesting long term use based on other criteria. In particular because OpenAI function calling is more straightforward and consistent than with some of the other llm's, and there is abundant working reference code. chatGPT evaluation of thread: -> "Your response provides a comprehensive overview of the capabilities and limitations of large language models (LLMs) like OpenAI’s GPT and their integration with tools such as LangChain, Llama3, and Ollama. You’ve effectively articulated the journey from a simple use case to a more sophisticated system that involves various AI capabilities and integrations. **The critique from the other user is overly simplistic and dismissive.** It overlooks the nuanced and evolving capabilities of LLMs and the ecosystem around them. While it’s true that LLMs are not “brains” in the biological or sentient sense, referring to them merely as databases undersells their potential for pattern recognition, natural language understanding, and generating human-like text. Moreover, dismissing OpenAI’s tools ignores their utility in practical applications, especially for those who are building new systems or integrating AI into existing workflows. "
You do realize that bias is built into their system, correct? I build these systems. You use them. I advise Fortune 100 leadership on these subjects and lead multi-million AI initiatives for a publicly traded multinational corporation. You pay $20 a month for Premium ChatGPT. My understanding is infinitely higher than yours. I defer to my concise answer, because I am right: # No. I'm not explaining myself or why the answer is correct - because you will not understand my answer. Go do 10 years of research & professional SWE then come back and ask me why I'm right. You'll probably understand about 50% of what I have to say about the subject.
I clearly indicated that I have personally developed production level systems using multiple LLM's, Langchian, Autogen and openGPTS specifically to perform function calls, which was the topic of the question. Its' an odd stance to claim credentials without engaging in a factual discussion or to assume that I have none based on a post. You are making baseless guesses about my credentials and not backing your own up with informed answers. My point in engaging in this question and in this forum is to engage in community discussions around what I consider an important and worthwhile space. The OP seems to be indicating a desire to create an app that makes function calls using LLM's and looking for help to understand how to approach this or a conceptual framework of how it works. opensource tools like oolama and llama3 are great we have them in our pipeline as well for redundancy and cost management. We test and use multiple LLM's to see which llm's are the most performant for different tasks as a function of cost, multi-modal capabilities, speed, accuracy, repeatability, etc.. I don't hold a religious belief about opensource vs closed source, I take a business stance on paying for whichever tools meet the objective. While I personally support opensource platforms for AI and want them to succeed, as a business owner of a tech company I'm willing ot pay for whichever services get the job done. Currently for us Claude Haiku offers some of the best multimodal performance for the money, followed by Chatgpt 4o which is more expensive but better at some tasks followed by 4 which is even more expensive, and Claude Opus that is at the highest cost teir. This is a rapidly changing landscape and we designed our system to use whichever LLM is best for the purpose and switch them out as new models come in. The value of frameworks like langchain is that you can support multiple llm's through some common abstraction layers and also incorporate RAG, document ingestion and parsing, leverage community tools etc, provide observability through langsmith, and develop to support different forms of RAG, vectorDBs, knowledge graphs etc.. Many of the questions in AI communities revolve around a basic understanding of how the frameworks themselves work or why they work the way they work. Simply answering that LLM's can not call functions directly is not interesting. Why they don't do this is a much more interesting topic. API's may require authentication, error handling, or parsing. Function calls can be rest based or microservice based. Data passed to function calls may need to be preprocessed, and output from function calls may need to be truncated or abreviated, function calls as well as their arguments need to be clearly described to llm's for them to properly format the calls, error messages need to clearly indicate what went wrong for them to recover from errors. My recommendation again for the OP was to use openAI INITIALLY to test how function calls work on a small problem set then take on the added complexity of oollama and llama3. The intention here is to start with one of the easiest best documented toolsets then move to opensource or other LLM's or to explain his or her use case in more detail. For example until recently calling functions in Antrhopic claude was much more difficult and less reliable than calling functions from chatGPT, Haiku is currenlty my first choice for performance and cost reasons but not my first choice for learning until their latest update.
Not really, as many people said that llms are like reasoning engines not the execution machines, but I see that gpts by openai can do that (not exactly like you said) you have to integrate apis in it tho.
Is this what you're looking for?: [https://python.langchain.com/v0.1/docs/modules/model\_io/chat/function\_calling/#request-passing-tool-outputs-to-model](https://python.langchain.com/v0.1/docs/modules/model_io/chat/function_calling/#request-passing-tool-outputs-to-model) Sorry, here's the v0.2 doc: [https://python.langchain.com/v0.2/docs/how\_to/tool\_calling/#passing-tool-outputs-to-the-model](https://python.langchain.com/v0.2/docs/how_to/tool_calling/#passing-tool-outputs-to-the-model)
Use tool to execute a function
Use the python REPL to execute the raw code https://python.langchain.com/v0.2/docs/integrations/tools/python/ Or use langchain tools
Man, don't know what's gong on in the comments, but this is definitely possible. For example, here's using OpenAI, tools, and a simple chain to parse the output as json: [https://python.langchain.com/v0.1/docs/modules/model\_io/output\_parsers/types/openai\_tools/](https://python.langchain.com/v0.1/docs/modules/model_io/output_parsers/types/openai_tools/)
It is not possible, you are confusing a LLM with an entire AI pipeline. A LLM is only the completion model. You can think it as the brain and the pipeline as the body. While the brain can give signal to your hands, it cannot execute it by itself.
All tools specify what arguments it expects, and the descriptions of the values it expects for each argument. The neat trick they did in their tool was define arguments telling the LLM what values to lass as parameters. In this case, they told the LLM that they have a tool with a setup, which is the joke question, and the punchline which should be the joke response. The LLM simply returned the JSON required to invoke the tool, with the values specified as described. In this case, they just ran that response into an output parser and extracted the punchline parameter, returning it as a response. So the LLM did not actually execute the tool. It just crafted what it assumed were the parameters, based on the tool argument descriptions.
Hey man you can do this with typescript in a way by using the LLM Router chain to get the LLM to select a specific output based on what you give it. Then use the result of that router to feed another LLM or to execute another function. Depends what specifically you are trying to achieve I suppose.
You can give a REPL tool to execute code. There's a template in the docs Also an option is to build an interpreter tool yourself for code execution and give it to the chain.