Typically a tf optimizer flow is as follows:
# Create an optimizer.
opt = GradientDescentOptimizer(learning_rate=0.1)
# Compute the gradients for a list of variables.
grads_and_vars = opt.compute_gradients(loss, <list of variables>)
# grads_and_vars is a list of tuples (gradient, variable). Do
# whatever you
# need to the 'gradient' part, for example cap them, etc.
capped_grads_and_vars = [(MyCapper(gv[0]), gv[1]) for gv in grads_and_vars]
# Ask the optimizer to apply the capped gradients.
opt.apply_gradients(capped_grads_and_vars)
How can I modify this flow so as loss function is computed using different datapoints after each optimizer step and these datapoints are computed as a function of the updated parameters?
Say that I have a stochastic process which has a certain parameterization and parameter values are learned by gradient descent. After updating the parameter values, I have to resample points from this process and evaluate the likelihood on these points (instead of the points I used in my previous step). Can I easily incorporate this in a typical tf optimizer? Conceptually, is similar to batch gradient descent, however datapoints of each batch are generated over the gradient descent loop in a way that depends on the parameter values of the previous iteration.