Writing your first LLM app

In this tutorial, we'll lead you through the process of creating your first Language Learning Model (LLM) application using Agenta. Our aim is to build an application capable of producing a compelling startup pitch based on the startup's name and core idea. By the end of the tutorial, we'll have our app set up locally, ready for iterative refinement in the playground. You'll have the ability to modify parameters, branch out new variants, and systematically assess different versions in Agenta.

Let's begin.

Installation

Run pip install agenta to install the Agenta CLI.

1. Project Initialization

Start by creating an empty project in an empty directory.

mkdir my-first-app; cd my-first-app
agenta init

Follow the prompts to initialize your project. Make sure to select start with an empty project when prompted.

Now create a virtual environment:

python3 -m venv env
source env/bin/activate

Finally let's create a requirements.txt and add within it our dependencies

langchain
agenta
python-dotenv
openai

Install the dependencies:

pip install -r requirements.txt

Now we're ready to start writing our app.

2. Write a Simple LLM App

Create a file called app.py - the main script that Agenta SDK expects. This simple script takes a startup name and idea, and generates a pitch using langchain:

from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate

startup_name = "agenta"
startup_idea = "the open-source developer-first llmops platform"

default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""

#template is a float, it is used to decide what sampling temperature to use. Default to 0.7
temperature = 0.5

llm = OpenAI(temperature=temperature)
prompt = PromptTemplate(
    input_variables=["startup_name", "startup_idea"],
    template=prompt_template)

chain = LLMChain(llm=llm, prompt=prompt)
output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
print(output)

3. Encapsulate Your Code in a Function

Wrap your code into a function named generate. This function will be the primary function Agenta calls, taking in strings as inputs and returning a string as an output:

from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


def generate(startup_name, startup_idea):
    #template is a float, it is used to decide what sampling temperature to use. Default to 0.7
    temperature = 0.5
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output

4. Add the decorator

We'll now transform our plain Python function into an Agenta app. Use the Agenta decorator @post for this purpose. The decorator tells Agenta that this function is the main function in the app, and it specifies the inputs (and later parameters) that the app takes.

Note here that we made sure to add the type annotations to the function arguments. This is required for Agenta to be able to understand the inputs and outputs of the function.

from agenta import post
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
import os


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


@post
def generate(startup_name: str, startup_idea: str) -> str:
    #template is a float, it is used to decide what sampling temperature to use. Default to 0.7
    temperature = 0.5
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output

5. Parameter Addition

To iterate and fine-tune this app in the playground, we need to expose parameters. Accomplish this by adding additional arguments in the app function:

from agenta import post, TextParam, FloatParam
from agenta.types import MultipleChoiceParam
from dotenv import load_dotenv
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    llm = OpenAI(temperature=temperature)
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    chain = LLMChain(llm=llm, prompt=prompt)
    output = chain.run(startup_name=startup_name, startup_idea=startup_idea)
    return output

In this example, startup_name and startup_idea are required inputs, while prompt, temperature and model are optional parameters with default values. The types TextParam, FloatParam and MultipleChoiceParam are used to indicate these are parameters rather than inputs, and they will be made available for tuning in the Agenta UI. The type MultipleChoiceParam will allow the user to select from multiple choices of models, by default the model to use is text-davinci-003 since a default model to use wasn't specified. To specify a default model, modify the parameter to this:

model: MultipleChoiceParam = MultipleChoiceParam(
    "text-davinci-003",
    ["gpt-3.5-turbo", "gpt-4"]
)

This will result to having the app use the text-davinci-003 model.

6. Text Generation Function - call_llm

The provided code snippet below defines a function call_llm that uses the chosen language model to generate text output based on the provided parameters. The function takes inputs such as model, temperature, and prompt, along with additional keyword arguments (kwargs) required for text generation.

If the chosen model is "text-davinci-003", it uses the OpenAI class from the LangChain library to run an LLMChain with the given parameters, producing text output based on the input prompt and the provided keyword arguments.

If the model is either "gpt-3.5-turbo" or "gpt-4", it uses the ChatOpenAI class from the LangChain library to generate text output in a conversational format. The function prepares a list of messages with a single HumanMessage containing the content from the input prompt formatted with the provided keyword arguments. The generated text output is then returned.

def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        messages = [
            HumanMessage(content=prompt.format(**kwargs))
        ]
        output = chat(messages).content
    return output

This function facilitates the text generation process using different language models, providing the necessary flexibility for users to choose the appropriate model for their specific needs.

7. Integration of the call_llm function

The call_llm function is seamlessly integrated into the generate app to provide text generation capabilities based on user inputs and selected language models.

# other imports here
from agenta.types import MultipleChoiceParam
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage


def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        message = HumanMessage(content=prompt.format(**kwargs))
        output = chat(message).content
    return output


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    output = call_llm(
        model=model, temperature=temperature, prompt=prompt,
        startup_name=startup_name, startup_idea=startup_idea
    )
    return output

The generate app function accepts parameters such as startup_name, startup_idea, prompt_template, temperature, and model. It initializes a PromptTemplate with startup_name and startup_idea as input variables, along with the provided prompt_template. The call_llm function is then invoked with the specified model, temperature, prompt, and keyword arguments (startup_name and startup_idea).

The function utilizes the chosen language model to generate a LinkedIn message based on the prompt_template and input data (startup_name and startup_idea). The generated LinkedIn message is returned as the output of the generate app.

NOTE: The call_llm function significantly enhances the capabilities of the generate app, providing a flexible and powerful mechanism for text generation with different language models. Its seamless integration within the app allows users to easily fine-tune text generation based on their specific requirements and chosen models.

8. API Key Integration

To use openai or other APIs, you'll need appropriate keys. Create an empty .env file and add your keys to it. Then, read the keys from the environment in your code:

OPENAI_API_KEY=sk-xxxxxxx

We just need to read the keys from the environement and use them in our code.

from agenta import post, TextParam, FloatParam
from dotenv import load_dotenv
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from agenta.types import MultipleChoiceParam
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage


default_prompt = """
    please write a short linkedin message (2 SENTENCES MAX) to an investor pitching the following startup:
    startup name: {startup_name}
    startup idea: {startup_idea}"""


def call_llm(model, temperature, prompt, **kwargs):
    if model == "text-davinci-003":
        llm = OpenAI(model=model, temperature=temperature)
        chain = LLMChain(llm=llm, prompt=prompt)
        output = chain.run(**kwargs)
    elif model in ["gpt-3.5-turbo", "gpt-4"]:
        chat = ChatOpenAI(model=model,
                          temperature=temperature)
        message = HumanMessage(content=prompt.format(**kwargs))
        output = chat(message).content
    return output


@post
def generate(
    startup_name: str,
    startup_idea: str,
    prompt_template: TextParam = default_prompt,
    temperature: FloatParam = 0.5,
    model: MultipleChoiceParam = MultipleChoiceParam(
        choices = ["text-davinci-003", "gpt-3.5-turbo", "gpt-4"]
    )
) -> str:
    load_dotenv()
    prompt = PromptTemplate(
        input_variables=["startup_name", "startup_idea"],
        template=prompt_template)

    output = call_llm(
        model=model, temperature=temperature, prompt=prompt,
        startup_name=startup_name, startup_idea=startup_idea
    )
    return output

9. Run the code in CLI

The @post decorator allows the function to run in the Command Line Interface (CLI). Execute it using the command:

python app.py agenta "the open-source developer-first llmops platform"

Hi there, I'm excited to tell you about Agenta, the open source Dev First LLMOPS Platform. It's a revolutionary way to simplify and streamline the development process, and I'm confident it will be a game changer in the industry. Let's chat soon!

If no default value was provided in the model (MultiChoiceParam) parameter, the code would still work because the app is using the first model in the list of models provided. If you would like to specify which model to use, kindly provide the name of the model and run the command again:

python app agenta "the open-source developer-first llmops platform" --model gpt-3.5-turbo

Hi [Investor's Name], I wanted to introduce you to Agenta, an open-source developer-first llmops platform. We believe it has the potential to revolutionize the way developers manage their llmops processes. Would love to discuss further if you're interested.

10. Deploying to Agenta

Our app is still local. To make it available in the playground, add it to Agenta by running:

agenta variant serve --file_name app.py

This commands build a docker image of your app and push it to the Agenta registry. It also creates a new variant for your app in the Agenta UI. You can now go to the Agenta UI and start playing with your app.

11. Iterative Improvement in the Playground

Your app is now accessible in the Agenta UI at http://localhost. Interact with your app, experiment with different parameters, and fine-tune your app's behavior to achieve desired results.

Conclusion

Congratulations! You've crafted and deployed your first LLM app with Agenta. You can now experiment and refine it further in the playground.