Skip to main content
TextGrad: Automatic "Differentiation" via TextMert Yuksekgonul et al.

Momentum

TEXTGRAD supports the use of Momentum in the Textual Gradient Descent. In standard SGD momentum uses a linear combination of past gradients and the most recent one to define a new gradient to update a variable. Similarly, TEXTGRAD keeps track of past iterations of the variable. This postfix is appended to the prompt for the optimizer.

1 optimizer = TextualGradientDescent(parameters=[solution], momentum_window=3)

Code Snippet 6: How to enable momentum using 3 previous steps in the TextualGradientDescent optimizer.

Momentum prompt

Here are the past iterations of this variable: &lt;PAST_ITERATIONS&gt; {past_values}</PAST_ITERATIONS>