The general case
The abstraction readily applies to arbitrarily complex systems. Define a computation graph by
where is a variable in the graph, is the set of all variables in the graph, and SuccessorsOf returns the successors and PredecessorsOf returns the predecessors of a variable. Generally speaking, the value of can be unstructured data, such as natural language text or images. For most of the results and exposition in this paper, is natural language text.
Further, let us have as the transformation that consumes a set of variables and produces the variable . For instance, we can use an LLM or Numerical Simulator as a transformation. Since different functions will have different ways to compute gradients and collect feedback for, we will generally use to denote the gradient function for a function . For the sake of exposition, we will omit the subscript when the function is obvious.
The gradients are computed by
where we collect the set of gradients from all successors of . Intuitively, we get feedback from every context in which a variable was used, and aggregate them.
Equation 11 recursively computes the gradients of the downstream objective with respect to the desired variables in the graph. The function takes as input the gradients of with respect to the successors of a given variable , the value of the variable and the successors themselves. Note that the final gradient variable comprises a set of contexts and criticisms for any place a variable was used in.
Finally, to update any desired variable in the graph, we can use an optimizer:
which updates the value of based on its current value and the gradients. For a computation graph where there are edges, each iteration of optimization performs at most additional language model calls to compute gradients (1 call using the gradient operator for each edge in the computation graph). For implementations of operations in TEXTGRAD, see Appendix A.