{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vnvlwUDZK7VA"
   },
   "source": [
    "## Demo Notebook of Function Calling with liteLLM\n",
    "- Supported Providers for Function Calling\n",
    "  - OpenAI - `gpt-4-0613` and `gpt-3.5-turbo-0613`\n",
    "- In this notebook we use function calling with `litellm.completion()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KrINCwRfLgZV"
   },
   "outputs": [],
   "source": [
    "## Install liteLLM\n",
    "!pip install litellm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "nK7zR5OgLlh2"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from litellm import completion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "dCQlyBxKLqbA"
   },
   "outputs": [],
   "source": [
    "os.environ['OPENAI_API_KEY'] = \"\" #@param"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gfdGv-FMRCdX"
   },
   "source": [
    "## Define Messages, Functions\n",
    "We create a get_current_weather() function and pass that to GPT 3.5\n",
    "\n",
    "See OpenAI docs for this: https://openai.com/blog/function-calling-and-other-api-updates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "ERzsP1sfM19C"
   },
   "outputs": [],
   "source": [
    "messages = [\n",
    "    {\"role\": \"user\", \"content\": \"What is the weather like in Boston?\"}\n",
    "]\n",
    "\n",
    "def get_current_weather(location):\n",
    "  if location == \"Boston, MA\":\n",
    "    return \"The weather is 12F\"\n",
    "\n",
    "functions = [\n",
    "    {\n",
    "      \"name\": \"get_current_weather\",\n",
    "      \"description\": \"Get the current weather in a given location\",\n",
    "      \"parameters\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "          \"location\": {\n",
    "            \"type\": \"string\",\n",
    "            \"description\": \"The city and state, e.g. San Francisco, CA\"\n",
    "          },\n",
    "          \"unit\": {\n",
    "            \"type\": \"string\",\n",
    "            \"enum\": [\"celsius\", \"fahrenheit\"]\n",
    "          }\n",
    "        },\n",
    "        \"required\": [\"location\"]\n",
    "      }\n",
    "    }\n",
    "  ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NX6by2VuRPnp"
   },
   "source": [
    "## Call gpt-3.5-turbo-0613 to Decide what Function to call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QVoJ5PtxMlVx",
    "outputId": "efe7a81f-e04a-4afc-aa60-a2b2648f5fb9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"id\": \"chatcmpl-7mX4RiqdoislVEqfmfVjFSKp3hyIy\",\n",
      "  \"object\": \"chat.completion\",\n",
      "  \"created\": 1691801223,\n",
      "  \"model\": \"gpt-3.5-turbo-0613\",\n",
      "  \"choices\": [\n",
      "    {\n",
      "      \"index\": 0,\n",
      "      \"message\": {\n",
      "        \"role\": \"assistant\",\n",
      "        \"content\": null,\n",
      "        \"function_call\": {\n",
      "          \"name\": \"get_current_weather\",\n",
      "          \"arguments\": \"{\\n  \\\"location\\\": \\\"Boston, MA\\\"\\n}\"\n",
      "        }\n",
      "      },\n",
      "      \"finish_reason\": \"function_call\"\n",
      "    }\n",
      "  ],\n",
      "  \"usage\": {\n",
      "    \"prompt_tokens\": 82,\n",
      "    \"completion_tokens\": 18,\n",
      "    \"total_tokens\": 100\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "response = completion(model=\"gpt-3.5-turbo-0613\", messages=messages, functions=functions)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yu0o2saDNLx8"
   },
   "source": [
    "## Parse GPT 3.5 Response\n",
    "Read Information about what Function to Call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u1DzXLJsNOR5",
    "outputId": "177e9501-0ce2-4619-9067-3047f18f6c79"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<OpenAIObject at 0x7922c70ce930> JSON: {\n",
       "  \"name\": \"get_current_weather\",\n",
       "  \"arguments\": \"{\\n  \\\"location\\\": \\\"Boston, MA\\\"\\n}\"\n",
       "}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function_call_data = response[\"choices\"][0][\"message\"][\"function_call\"]\n",
    "function_call_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tYb96Mh0NhH9",
    "outputId": "13c4bb89-6f29-4b3b-afa7-302dcf2cdd5f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "get_current_weather {'location': 'Boston, MA'}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "function_name = function_call_data['name']\n",
    "function_args = function_call_data['arguments']\n",
    "function_args = json.loads(function_args)\n",
    "print(function_name, function_args)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z3tstH_yN3fX"
   },
   "source": [
    "## Call the get_current_weather() function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TSb8JHhgN5Zc",
    "outputId": "ef140572-4020-4daf-ac8c-d5161be9aa5c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12F\n"
     ]
    }
   ],
   "source": [
    "if function_name == \"get_current_weather\":\n",
    "  result = get_current_weather(**function_args)\n",
    "  print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k4HGJE3NRmMI"
   },
   "source": [
    "## Send the response from get_current_weather back to the model to summarize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "a23cmEwiPaw7",
    "outputId": "43259b86-0c4c-4fcb-eab7-6e1a788b2f21"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"id\": \"chatcmpl-7mXGN62u75WXp1Lgen4iSgNvA7hHT\",\n",
      "  \"object\": \"chat.completion\",\n",
      "  \"created\": 1691801963,\n",
      "  \"model\": \"gpt-3.5-turbo-0613\",\n",
      "  \"choices\": [\n",
      "    {\n",
      "      \"index\": 0,\n",
      "      \"message\": {\n",
      "        \"role\": \"assistant\",\n",
      "        \"content\": \"The current weather in Boston is 12 degrees Fahrenheit.\"\n",
      "      },\n",
      "      \"finish_reason\": \"stop\"\n",
      "    }\n",
      "  ],\n",
      "  \"usage\": {\n",
      "    \"prompt_tokens\": 109,\n",
      "    \"completion_tokens\": 12,\n",
      "    \"total_tokens\": 121\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "messages = [\n",
    "    {\"role\": \"user\", \"content\": \"What is the weather like in Boston?\"},\n",
    "    {\"role\": \"assistant\", \"content\": None, \"function_call\": {\"name\": \"get_current_weather\", \"arguments\": \"{ \\\"location\\\": \\\"Boston, MA\\\"}\"}},\n",
    "    {\"role\": \"function\", \"name\": \"get_current_weather\", \"content\": result}\n",
    "]\n",
    "response = completion(model=\"gpt-3.5-turbo-0613\", messages=messages, functions=functions)\n",
    "print(response)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}