What Does It Mean That A Tf.variable Is Trainable In Tensorflow

This question came to me when I read the documentation of global_step. Here it explicitly declares global_step is not trainable. global_step_tensor = tf.Variable(10, trainable=Fal

Solution 1:

From my understanding, trainable means that the value could be changed during sess.run()

That is not the definition of a trainable variable. Any variable can be modified during a sess.run() (That's why they are variables and not constants).

The distinction between trainable variables and non-trainable variables is used to let Optimizers know which variables they can act upon. When defining a tf.Variable(), setting trainable=True (the default) automatically adds the variable to the GraphKeys.TRAINABLE_VARIABLES collection. During training, an optimizer gets the content of that collection via tf.trainable_variables() and applies the training to all of them.

The typical example of a non-trainable variable is global_step, because its value does change over time (+1 at each training iteration, typically), but you don't want to apply an optimization algorithm to it.