Warmup: system with two LLM calls
Example computation graph. In traditional automatic differentiation, we compute gradients that provides a direction that would improve the variable with respect to a downstream loss with the chain- rule. For a simple analog, let us look at the simple system:
a Neural network and backpropagation using numerical gradients

b Blackbox AI systems and backpropagation using natural language 'gradients'

C 1 Analogy in abstractions
| Math | PyTorch | TextGrad | |
| Input | x | Tensor(image) | tg.Variable(article) |
| Model | y = fθ(x) | ResNet50( ) | tg.BlackboxLLM("You are a summarizer.") |
| Loss | L(y, y) = ∑ y; log( y; ) | CrossEntropyLoss( ) | tg.TextLoss("Rate the summary.") |
| Optimizer | GD(θ; θ) = θ - θL/θ0 | SGD(List(model.parameters( ))) | tg.TGD(List(model.parameters( ))) |
2 Automatic differentiation
PyTorch and TextGrad share the same syntax for backpropagation and optimization.

d TextGrad for molecule optimization

e TextGrad for code optimization



f TextGrad for treatment plan optimization
g TextGrad for prompt optimization
You will answer a reasoning question. Think step by step. The last line of your response should be of the following format: 'Answer: 'SVAULE' where VALUEl is a numerical value.
Prompt at initialization (Accuracy = 77.8%)
You will answer a reasoning question. List each item and its quantity in a clear and consistent format, such as - Item: Quantity'. Sum the values directly from the list and provide a concise summation. Ensure the final answer is clearly indicated in the format: 'Answer: 'SVAULE' where VALUEl is a numerical value. Verify the relevance of each item to the context of the query and handle potential errors or ambiguities in the input. Double-check the final count to ensure accuracy."
Prompt after optimization (Accuracy = 91.9%)
Figure 1: Automatic "Differentiation" via Text (a, b). Backpropagation of gradients is the driving force of deep learning. We do not have gradients for compound systems of blackbox AI systems, but we can construct analogous backpropagation for text- based feedback, forming the basis of TEXTGRAD. Abstractions in TEXTGRAD (c). We share the same abstractions and syntax as PyTorch to our framework generalizable and easy- to- learn. Applications of TEXTGRAD (d,e,f,g). In Section 3.4, we optimize molecular structures for properties such as druglikeness and protein binding affinity (d). In Section 3.1, we optimize solutions to coding problems (e). In Section 3.5, we optimize radiotherapy treatment plans to improve patient outcomes (f). In Section 3.3, we optimize prompts to improve the reasoning of language models (g). where we denote the free parameter to optimize, the prompt, with green, and we use to denote concatenation of two strings, and use to give as a prompt to a language model to collect the response. We will use the chain notation:
as another way to denote this system, where we make one LLM call to generate a Prediction for a Question using a Prompt, and another LLM call to evaluate this Prediction.
Example gradient computation. In this example system, to improve the Prompt with respect to the evaluation, we instantiate an analog of the backpropagation [19] algorithm through the following:
Gradient computation for the simple graph: Prompt Prediction Evaluation
where we use for the gradient operator when the forward function is an LLM call. In particular, this function returns natural language feedback such as 'This prediction can be improved by...' where the feedback describes how to modify the variable to improve the downstream objective, analogous to gradients in optimization. We first collect feedback to the Prediction variable using the evaluation. Then, given this feedback and the (Prompt Prediction) call, we collect the feedback on the Prompt.
"Note that we overload the notation to denote both for cases when the output variable does not have successors (e.g., ) and when it has successors, and thus gradients (e.g., ).
Below is a flexible way to instantiate to collect feedback for a simple system like :
Example implementation for the gradient operator
where the gradient object is a combination of the context in which the variable appears and the feedback obtained from an , defined analogously to Pryzant et al. [25]. Note that this operator does not depend on, e.g., the application domain. Once implemented, the gradient operator for LLM calls are fixed throughout the framework for all applications.
aThe exact prompts we use are different to ensure generality and flexibility; we use these examples only for exposition.
Example optimizer. In standard gradient descent, the current value of the variable is combined with the gradients through subtraction, e.g.:
Continuing the gradient- based optimization analogy, we use Textual Gradient Descent (TGD):
Updating the Prompt variable in the simple graph via TGD.
to update the parameters, in this case, the Prompt. In particular, given the current variable and the gradients (feedback) we collected for this variable, the optimizer seeks to update this variable.
A concrete way to instantiate TGD is the following:
Example implementation of one TGD iteration
where is the variable we would like to improve, and is the feedback we obtained for the variable during the backward passa. Similar to the gradient operator, this function also does not depend on the domain of application, and TGD implementation is the same across all uses of the framework.
aThe exact prompts we use are different to ensure generality and flexibility; we use these examples only for exposition.