The Langchain Interface : Chains and Runnables

Photo by Possessed Photography on Unsplash

To make it easy to create custom chains, Langchain uses a Runnable protocol. It is a standard interface which makes it easy to define and invoke custom chains in a standard way.

They include:

stream which streams back chunks of the response.

invoke which calls the chain on a single input

batch which calls the chain on a list of inputs

These also have corresponding asynchronous methods.

Learn more here: https://python.langchain.com/docs/expression_language/interface

Different components have different inputs and outputs.

All runnables expose an input or output schemas which provides us the ability to inspect the inputs and outputs. i.e. input_schema and output_schema which are pydantic models autogenerated from the structure of the runnables.

Creating a simple chain

from langchain.chat_models import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

model = ChatOpenAI()
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")

chain = prompt | model

Input Schema

It is a description of the inputs that are accepted by the runnable. The input schema of the chain is the input schema of the first runnable.

In order to obtain the schema from a chain, we can;

# getting the input schema of the chain
chain_input_schema = chain.input_schema.schema()

#getting the input schema of the prompt and model
prompt_input_schema = prompt.input_schema.schema()
model_input_schema = model.input_schema.schema()

Output schema

A description of the output produced by the runnable. And correspondingly the output schema of the chain is the representation of the output of the last runnable within the chain.

chain_output_schema = chain.output_schema.schema()
prompt_output_schema = prompt.output_schema.schema()
model_output_schema = model.output_schema.schema()

Stream

This is a stream of chunks of the output of the response

# the input is a dictionary which is consumed by the prompt component of the chain
for s in chain.stream({"topic": "bear"}):
    print(s.content, end="", flush=True)

Invoke

This calls a chain on an input.

chain.invoke({"topic": "bears"})
# Output: AIMessage(content="Why don't bears wear shoes?\\n\\nBecause they already have bear feet!")

Batch

This calls a chain with a list of inputs

chain.batch([{"topic": "bears"}, {"topic": "cats"}])
"""
Output:     
[AIMessage(content="Why don't bears wear shoes?\\n\\nBecause they have bear feet!"),
AIMessage(content="Why don't cats play poker in the wild?\\n\\nToo many cheetahs!")]
"""

We can also set the number of concurrent requests by using the max_concurrency parameter

chain.batch([{"topic": "bears"}, {"topic": "cats"}], config = {"max_concurrency":5})

Parallelism

We can use RunnableParallel to execute more than one chain in parallel.

chain1 = ChatPromptTemplate.from_template("tell me a joke about {topic}") | model
chain2 = (
    ChatPromptTemplate.from_template("write a short (2 line) poem about {topic}")
    | model
)

combined = RunnableParallel(joke=chain1,bears=chain2)

#And we can invoke the runnable normally using the invoke method
combined.invoke({"topic":"bears"})

"""
Output:
        CPU times: user 167 ms, sys: 921 µs, total: 168 ms
    Wall time: 1.56 s
    {'joke': AIMessage(content="Why don't bears wear shoes?\n\nBecause they already have bear feet!"),
     'poem': AIMessage(content="Fierce and wild, nature's might,\nBears roam the woods, shadows of the night.")}
"""

Parallelism can also be used with batches

combined.batch([{"topic": "bears"}, {"topic": "cats"}])

"""
Output:
    CPU times: user 507 ms, sys: 125 ms, total: 632 ms
    Wall time: 1.49 s

    [{'joke': AIMessage(content="Why don't bears wear shoes?\n\nBecause they already have bear feet!"),
      'poem': AIMessage(content="Majestic bears roam,\nNature's wild guardians of home.")},
     {'joke': AIMessage(content="Sure, here's a cat joke for you:\n\nWhy did the cat sit on the computer?\n\nBecause it wanted to keep an eye on the mouse!"),
      'poem': AIMessage(content='Whiskers twitch, eyes gleam,\nGraceful creatures, feline dream.')}]
"""