Skip to main content
TextGrad: Automatic "Differentiation" via TextMert Yuksekgonul et al.

The general case

The abstraction readily applies to arbitrarily complex systems. Define a computation graph by

v=fv(PredecessorsOf(v))∀v∈V(10)v = f_{v}(\mathrm{PredecessorsOf}(v))\forall v\in \mathcal{V} \quad (10)

where vv is a variable in the graph, V\mathcal{V} is the set of all variables in the graph, and SuccessorsOf returns the successors and PredecessorsOf returns the predecessors of a variable. Generally speaking, the value of vv can be unstructured data, such as natural language text or images. For most of the results and exposition in this paper, vv is natural language text.

Further, let us have fvf_{v} as the transformation that consumes a set of variables and produces the variable vv . For instance, we can use an LLM or Numerical Simulator as a transformation. Since different functions will have different ways to compute gradients and collect feedback for, we will generally use ∇f\nabla_{f} to denote the gradient function for a function ff . For the sake of exposition, we will omit the subscript when the function is obvious.

The gradients are computed by

∂L∂v=⋃w∈SuccessorsOf(v)∇f(v,w,∂L∂w),(11)\frac{\partial\mathcal{L}}{\partial v} = \bigcup_{w\in \mathrm{SuccessorsOf}(v)}\nabla_{\mathrm{f}}\left(v,w,\frac{\partial\mathcal{L}}{\partial w}\right), \quad (11)

where we collect the set of gradients from all successors of vv . Intuitively, we get feedback from every context in which a variable vv was used, and aggregate them.

Equation 11 recursively computes the gradients of the downstream objective with respect to the desired variables vv in the graph. The ∇i\nabla_{i} function takes as input the gradients of L\mathcal{L} with respect to the successors of a given variable vv , the value of the variable vv and the successors themselves. Note that the final gradient variable comprises a set of contexts and criticisms for any place a variable was used in.

Finally, to update any desired variable vv in the graph, we can use an optimizer:

vnew=TGD.step(v,∂L∂v),(12)v_{\mathrm{new}} = \mathrm{TGD.step}\left(v,\frac{\partial\mathcal{L}}{\partial v}\right), \quad (12)

which updates the value of vv based on its current value and the gradients. For a computation graph where there are nn edges, each iteration of optimization performs at most nn additional language model calls to compute gradients (1 call using the gradient operator for each edge in the computation graph). For implementations of operations in TEXTGRAD, see Appendix A.