Skip to main content
TextGrad: Automatic "Differentiation" via TextMert Yuksekgonul et al.

A.3 Functions

TEXTGRAD offers several operations where both the forward and backward computations are defined – as such, these operations are composable. The abstract textgrad.autograd.Function class has two methods:

Algorithm 1 Backpropagation in TEXTGRAD

1: Input: Variables v∈Vv\in \mathcal{V} in a graph, Loss variable L\mathcal{L} , Backward Engine (LLM) M\mathcal{M} that will provide textual gradients 2: # Initializing gradients 3: for each v∈Vv\in \mathcal{V} do 4: vv gradients =∅= \emptyset 5: end for 6: # Topological Sorting 7: Q←Q\leftarrow TopologicalSort(G) 8: # Backpropagation 9: for vv in QQ do 10: # Populate gradients in predecessors 11: for each u∈u\in PredecessorsOf(v) do 12: # Here, we are omitting subscript vv in ff . Semantically, ff is the function that generates vv , and ∇f\nabla_{f} is the backward operation for that function. 13: # Semantically, this provides feedback to the variable uu , given how vv is produced, and the feedback we already collected for vv . 14: uu .gradients.add (∇f(u,v,∂L∂v))\left(\nabla_{f}\left(u,v,\frac{\partial\mathcal{L}}{\partial v}\right)\right) 15: end for 16: end for forward and backward, mirroring the PyTorch syntax. Each function has to define both of these methods. Below, we describe a couple of the most used operations in this paper.

LLMCall Function: Currently, the most crucial operation in TEXTGRAD is the call to language models.

Forward mode. The forward mode is simple: We make a call to an LLM, through an API or through the local machine. When a call is made, all the input variables are registered as the predecessors of the response from the LLM. For instance, if we ask a question to an LLM using an instruction and a question variable, the response variable's predecessors will be [instruction, question]. When doing the backward pass, the gradients on the response will be backpropagated to question and instruction.

Backward mode. To ensure the backward function runs smoothly and generally, we add the following glossary to the system prompt:

Glossary for Backward Mode of the LLMCall function

### Glossary of tags that will be sent to you:- - <LM_SYSTEM_PROMPT>: The system prompt for the language model.- - <LM_INPUT>: The input to the language model.- - <LM_OUTPUT>: The output of the language model.- - <OBJECTIVE_FUNCTION>: The objective of the optimization task.- - <VARIABLE>: Specifies the span of the variable.- - <ROLE>: The role description of the variable.

Using these tags, the LLM is made aware of the objective, the role and the value of the variable to give feedback to, and the full conversation in the forward mode.

This glossary is provided in the system prompt to the backward engine LLM:

System prompt for the backward mode of the LLMcall function

You are part of an optimization system that improves a given text (i.e. the variable). You are the gradient (feedback) engine. Your only responsibility is to give intelligent and creative feedback and constructive criticism to variables, given an objective specified in <OBJECTIVE_FUNCTION </OBJECTIVE_FUNCTION> tags. The variables may be solutions to problems, prompts to language models, code, or any other text- based variable. Pay attention to the role description of the variable, and the context in which it is used. You should assume that the variable will be used in a similar context in the future. Only provide strategies, explanations, and methods to change in the variable. DO NOT propose a new version of the variable, that will be the job of the optimizer. Your only job is to send feedback and criticism (compute 'gradients'). For instance, feedback can be in the form of 'Since language models have the X failure mode...', 'Adding X can fix this error because...', 'Removing X can improve the objective function because...', 'Changing X to Y would fix the mistake ...', that gets at the downstream objective.

If a variable is already working well (e.g. the objective function is perfect, an evaluation shows the response is accurate), you should not give feedback.

[GLOSSARY]

Most of this setup aims to ensure that the user does not have to modify the gradient computation. All of our experiments in the diverse set of applications are done with the same backward mode.

For instance, an example prompt for the gradient computation looks like the following:

An example backward mode prompt for the LLMcall function

You will give feedback to a variable with the following role: &lt;ROLE&gt; system prompt to a language model &lt;/role&gt;. Here is a conversation with a language model (LM):

&lt;LM_SYSTEM_PROMPT&gt; You will answer a reasoning question. Think step by step. Always conclude the last line of your response should be of the following format: 'Answer: §\S VALUE' where VALUE is a numerical value.

</LM_SYSTEM_PROMPT>

&lt;LM_INPUT&gt; I have a chicken, a frog, a mouse, a cat, two pigs, and two rabbits. How many animals do I have? </LM_INPUT>

&lt;LM_OUTPUT&gt; To find the total number of animals, we simply add up the number of each type of animal: 1 chicken + 1 frog + 1 mouse + 1 cat + 2 pigs + 2 rabbits = 8 animals.

Answer: 8</LM_OUTPUT>8< /LM\_ OUTPUT>

This conversation is part of a larger system. The &lt;LM_OUTPUT&gt; was later used as response from the language model. &lt;OBJECTIVE_FUNCTION&gt; Your goal is to give feedback to the variable to address the following feedback on the LM_OUTPUT: To improve the runtime of the string- based function that checks if the prediction is correct, consider the following feedback:

1. \*\*Simplify the Response\*\*: For example, instead of "To find the total number of animals, we simply add (...) + 2 pigs + 2 rabbits = 8 animals. Answer: 8", a more concise response like "Total animals: 8" would be more efficient.

By implementing these strategies, the response from the language model can be optimized to improve the runtime of the string- based function that checks if the prediction is correct. </OBJECTIVE_FUNCTION>

We are interested in giving feedback to the system prompt to a language model. Specifically, give feedback to the following span of text:

&lt;VARIABLE&gt; You will answer a reasoning question. Think step by step. (...) following format: 'Answer: §\S VALUE' where VALUE is a numerical value. &lt;/variable&gt;

Given the above history, describe how the system prompt to a language model could be improved to improve the &lt;OBJECTIVE_FUNCTION&gt;. Be very creative, critical, and intelligent.

Addition Operation In numerical optimization, we have the following:

z=x+yw=t+z∂w∂x=∂w∂z∂z∂x=∂w∂z(19)\begin{array}{c}{z = x + y}\\ {w = t + z}\\ {\frac{\partial w}{\partial x} = \frac{\partial w}{\partial z}\frac{\partial z}{\partial x} = \frac{\partial w}{\partial z}} \end{array} \quad (19)

... ... ... ...

In numerical derivatives, due to the linearity of addition, we have:

∂∑iNLi∂x=∑iN∂Li∂x.(22)\frac{\partial\sum_{i}^{N}L_{i}}{\partial x} = \sum_{i}^{N}\frac{\partial L_{i}}{\partial x}. \quad (22)

In particular, the backward function for the addition operation passes the gradients on the output of the addition operation to its inputs.

tg.sum(L1,…,LN)="{L1}\n{L2}\n…{LN}"(23)\mathrm{tg.}\mathrm{sum}(L_{1},\ldots ,L_{N}) = " \{L_{1}\} \backslash n\{L_{2}\} \backslash n\ldots \{L_{N}\} " \quad (23)

Similarly, in TEXTGRAD, we have the tg.sum operation, that lets the gradients (feedback) on the output variable pass to the input variables.

∂tg.sum(L1,…,LN)∂x=tg.sum(∂L1∂x,…,∂LN∂x)="{∂L1∂x}\n…{∂LN∂x}"(24)\frac{\partial\mathrm{tg.}\mathrm{sum}(L_{1},\ldots,L_{N})}{\partial x} = \mathrm{tg.}\mathrm{sum}(\frac{\partial L_{1}}{\partial x},\ldots ,\frac{\partial L_{N}}{\partial x}) = "{\Bigl \{}{\frac{\partial L_{1}}{\partial x}}{\Bigr \} \backslash n\ldots \Bigl \{}{\frac{\partial L_{N}}{\partial x}}{\Bigr \} "} \quad (24)

where we use ++ to indicate concatenation.

Use Case: One canonical use case for the addition operation is the batch optimization case. In particular, we implement minibatch gradient descent when performing prompt optimization (Section 3.3).