Skip to main content
TextGrad: Automatic "Differentiation" via TextMert Yuksekgonul et al.

Constrained Optimization with Natural Language Constraints

In TEXTGRAD it is possible use constraints when optimizing variables. These constraints are all defined as natural language descriptions. For example, one can prompt optimizer to update the variable but to conclude its response with an answer during the update:

Example constrained optimization prompt

You must follow the following constraints: <CONSTRAINTS> 'The last line of your response should be of the following format: 'Answer: $LETTER' where LETTER is one of ABCD."</constraints>

In general, the constraint post- fix is appended to the optimizer's prompt, where the constraints are written within the <CONSTRAINTS> {constraint text} </constraints> tags.

In code, the user can simply pass in the constraints to the TGD optimizer:

1 constraints = ["The last line of your response should be of the following format: 'Answer: \(LETTER' where LETTER is one of ABCD."] 2 optimizer = TextualGradientDescent(parameters=[solution], constraints=constraints)

Code Snippet 5: An example use for constraints when updating the solution to a problem.