diff --git a/cookbook/structured_report_generation.ipynb b/cookbook/structured_report_generation.ipynb new file mode 100644 index 000000000..e147c21af --- /dev/null +++ b/cookbook/structured_report_generation.ipynb @@ -0,0 +1,1603 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "vscode": { + "languageId": "plaintext" + }, + "id": "nB0yIE5qfKkq" + }, + "source": [ + "# Structured Report Generation\n", + "[](https://console.brev.dev/launchable/deploy?launchableID=env-2qzwjeLSpDcjU68lNQyDcZwxmn9)\n", + "\n", + "Deploy with Launchables. Launchables are pre-configured, fully optimized environments that users can deploy with a single click.\n", + "\n", + "In this notebook, you will use the newest llama model, llama-3.3-70b, to generate a report on a given topic. You will use Langchain's LangGraph to build an Agent that takes in user-defined topics and structure, then plans the topics of the section indicated in the structure. Next, the Agent uses Tavily to do web search on the given topics and uses this information to write the sections and synthesize the final report.\n", + "\n", + "Rather than deploying the model locally, you leverage NVIDIA API Catalog by calling the model's NIM API Endpoint. As you don't need a GPU to run the model, you can run this notebook anywhere!\n", + "\n", + "You can find the original notebook on LangChain's GitHub [here](https://github.com/langchain-ai/report-mAIstro).\n", + "\n", + "Below is the architecture diagram.\n", + "\n", + "# \n", + "\n", + "\n", + "\n", + "The Agent takes in user defined topics and structure, then plans the topics of the section indicated in the structure. The Agent then uses Tavily to do web search on the given topics and uses this information to write the sections and synthesize the final report.\n", + "\n", + "\n", + "A two-phase approach is used for planning and research:\n", + "\n", + "\n", + "Phase 1 - Planning\n", + "- Analyzes user inputs\n", + "- Maps out report sections\n", + "\n", + "\n", + "Phase 2 - Research\n", + "- Conducts parallel web research via Tavily API\n", + "- Processes relevant data for each section\n", + "\n", + "\n", + "\n", + "The report is then written in a strategic sequence:\n", + "- Write research-based sections in parallel\n", + "- Write introductions, conclusions, and connect each of the sections\n", + "\n", + "\n", + "All sections maintain awareness of each other's content for consistency." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3mLEcH4efKk-" + }, + "source": [ + "# Content Overview\n", + ">[Prerequisites](#Prerequisites) \n", + ">[Writing Report Plan](#Writing-Report-Plan) \n", + ">[Research and Writing](#Research-and-Writing) \n", + ">[Write Single Section](#Write-Single-Section) \n", + ">[Validate Single Section](#Validate-Single-Section) \n", + ">[Write All Sections](#Write-All-Sections) \n", + ">[Validate Final Report](#Final-Report)\n", + "________________________\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zKE7UqdqfKk-" + }, + "source": [ + "## Prerequisites\n", + "\n", + "### Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "id": "-nyq12MGfKk-" + }, + "outputs": [], + "source": [ + "%%capture --no-stderr\n", + "%pip install --quiet -U langgraph langchain_community langchain_core tavily-python langchain_nvidia_ai_endpoints" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b_Zlx5RbfKk_" + }, + "source": [ + "## API Keys\n", + "Prior to getting started, you will need to create API Keys for the NVIDIA API Catalog, Tavily, and LangChain.\n", + "\n", + "- NVIDIA NIM Trial API Key\n", + " 1. Prior to getting started, you will need to create API Keys to access NVIDIA NIM trial hosted endpoints.\n", + " 2. If you don’t have an NVIDIA account, you will be asked to sign-up.\n", + " 3. Click [here](https://build.nvidia.com/meta/llama-3_3-70b-instruct?signin=true&api_key=true) to sign-in and get an API key\n", + "- LangChain\n", + " 1. Go to **[LangChain Settings page](https://smith.langchain.com/settings)**. You will need to create an account if you have not already.\n", + " 2. On the left panel, navigate to \"API Keys\".\n", + " 3. Click on the \"Create API Key\" on the top right of the page.\n", + "- Tavily\n", + " 1. Go to the **[Tavily homepage](https://tavily.com/)** and click on \"Get Started\"\n", + " 2. Sign in or create an account.\n", + " 3. Create an API Key.\n", + "\n", + "### Export API Keys\n", + "\n", + "Save these API Keys as environment variables.\n", + "\n", + "First, set the NVIDIA API Key as the environment variable." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "scrolled": true, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "0lWPnmq1fKk_", + "outputId": "ff1f737a-1617-4298-a5bd-3c353bc1fcb7" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Enter your NVIDIA API key: ··········\n" + ] + } + ], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n", + " nvapi_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n", + " assert nvapi_key.startswith(\"nvapi-\"), f\"{nvapi_key[:5]}... is not a valid key\"\n", + " os.environ[\"NVIDIA_API_KEY\"] = nvapi_key" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "vscode": { + "languageId": "plaintext" + }, + "id": "oXdWXhIZfKk_" + }, + "source": [ + "Next, set the LangChain API Key as an environment variable. You will use [LangSmith](https://docs.smith.langchain.com/) for [tracing](https://docs.smith.langchain.com/concepts/tracing)." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "9LMDW_36fKk_", + "outputId": "e35ad86f-f050-4598-da1c-68b1e5210064" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LANGCHAIN_API_KEY: ··········\n" + ] + } + ], + "source": [ + "import os, getpass\n", + "\n", + "def _set_env(var: str):\n", + " if not os.environ.get(var):\n", + " os.environ[var] = getpass.getpass(f\"{var}: \")\n", + "\n", + "_set_env(\"LANGCHAIN_API_KEY\")\n", + "os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n", + "os.environ[\"LANGCHAIN_PROJECT\"] = \"report-mAIstro\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Bs3CG-rNfKk_" + }, + "source": [ + "Finally, set the Tavily API Key as an environment variable. You will use [Tavily API](https://tavily.com/) web search tool." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "_oNAkhtefKk_", + "outputId": "e6c61383-9070-4d91-adcc-c3d0ccf05483" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "TAVILY_API_KEY: ··········\n" + ] + } + ], + "source": [ + "_set_env(\"TAVILY_API_KEY\")" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "id": "-9S-0BlzfKk_" + }, + "outputs": [], + "source": [ + "from tavily import TavilyClient, AsyncTavilyClient\n", + "tavily_client = TavilyClient()\n", + "tavily_async_client = AsyncTavilyClient()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cZBUlvcdfKk_" + }, + "source": [ + "### Working with the NVIDIA API Catalog\n", + "\n", + "Let's test the API endpoint.\n", + "\n", + "In this notebook, you will use the newest llama model, llama-3.3-70b-instruct, as the LLM. Define the LLM below and test the API Catalog.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-GIhfQyZfKk_", + "outputId": "f26b2dfc-efca-4952-b1bd-6fa0fd883da9" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "(Verse 1)\n", + "In realms of code, where AI does roam\n", + "A framework emerged, to make it back home\n", + "LangChain, a name that's known so well\n", + "A tool for builders, to stories to tell\n", + "\n", + "(Chorus)\n", + "Oh LangChain, oh LangChain, a chain of might\n", + "Connecting languages, in the dark of night\n", + "A bridge of code, that spans so wide\n", + "Helping developers, to reach the other side\n", + "\n", + "(Verse 2)\n", + "With LLMs and APIs, it does abide\n", + "A hub of knowledge, where data does reside\n", + "It weaves a tapestry, of language and art\n", + "A symphony of code, that beats within the heart\n", + "\n", + "(Chorus)\n", + "Oh LangChain, oh LangChain, a chain of might\n", + "Connecting languages, in the dark of night\n", + "A bridge of code, that spans so wide\n", + "Helping developers, to reach the other side\n", + "\n", + "(Verse 3)\n", + "From Python to JavaScript, it does stride\n", + "A path of integration, where all can reside\n", + "It breaks the barriers, of language and space\n", + "A unified platform, where innovation takes its place\n", + "\n", + "(Chorus)\n", + "Oh LangChain, oh LangChain, a chain of might\n", + "Connecting languages, in the dark of night\n", + "A bridge of code, that spans so wide\n", + "Helping developers, to reach the other side\n", + "\n", + "(Bridge)\n", + "With every line, of code that's written\n", + "A new world unfolds, where possibilities are smitten\n", + "The future is bright, with LangChain in sight\n", + "A guiding light, that shines through the dark of night\n", + "\n", + "(Verse 4)\n", + "The community grows, with each passing day\n", + "A collective effort, to drive the framework on its way\n", + "With every contribution, a new story's told\n", + "Of LangChain's power, to make the impossible unfold\n", + "\n", + "(Chorus)\n", + "Oh LangChain, oh LangChain, a chain of might\n", + "Connecting languages, in the dark of night\n", + "A bridge of code, that spans so wide\n", + "Helping developers, to reach the other side\n", + "\n", + "(Outro)\n", + "So let the code flow, like a river's stream\n", + "With LangChain as the guide, the future's dream\n", + "A world of possibility, where all can thrive\n", + "With LangChain, the chain that connects, and survives.\n" + ] + } + ], + "source": [ + "## Core LC Chat Interface\n", + "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n", + "\n", + "llm = ChatNVIDIA(model=\"meta/llama-3.3-70b-instruct\", temperature=0)\n", + "result = llm.invoke(\"Write a ballad about LangChain.\")\n", + "print(result.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yC5c61XbfKk_" + }, + "source": [ + "### Optional: Locally Run NVIDIA NIM Microservices" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IA-KDt1xfKk_" + }, + "source": [ + "Once you familiarize yourself with this blueprint, you may want to self-host models with NVIDIA NIM Microservices using NVIDIA AI Enterprise software license. This gives you the ability to run models anywhere, giving you ownership of your customizations and full control of your intellectual property (IP) and AI applications.\n", + "\n", + "[Learn more about NIM Microservices](https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/)\n", + "\n", + "