graph_training_utils

Utilities for training the parameters of tensorflow computational graphs.

class graph_training_utils.EarlyStop(badlimit=20)[source]

A class for determining when to stop a training while loop by a bad count criterion. If the data is exhausted or the model’s performance hasn’t improved for badlimit training steps, the __call__ function returns false. Otherwise it returns true.

Parameters:badlimit – Limit of for number of training steps without improvement for early stopping.
__call__(mat, loss)[source]

Returns a boolean for customizable stopping criterion. For first loop iteration set loss to sys.float_info.max.

Parameters:
  • mat – Current batch of features for training.
  • loss – Current loss during training.
Returns:

boolean, True when mat is not None and self.badcount < self.badlimit and loss != inf, nan.
class graph_training_utils.ModelRunner(loss, ph_dict, learnrate=0.01, opt='adam', debug=False, decay=True, decay_rate=0.99, decay_steps=20)[source]

A class for gradient descent training tensorflow models.

Parameters:
  • loss – The objective function for optimization strategy.
  • ph_dict – A dictionary of names (str) to tensorflow placeholders.
  • learnrate – The step size for gradient descent.
  • opt – Optimization algorithm can be ‘adam’, or ‘grad’
  • debug – Whether or not to print debugging info.
  • decay – (boolean) Whether or not to use a learn rate with exponential decay.
  • decay_rate – The rate parameter for exponential decay of learn rate.
  • decay_steps – The number of training steps to decay learn rate.
eval(datadict, eval_tensors)[source]

Evaluates tensors without effecting parameters of model.

Parameters:
  • datadict – A dictionary of names (str) matching names in ph_dict to numpy matrices for this mini-batch.
  • eval_tensors – Tensors from computational graph to evaluate as numpy matrices.
Returns:

A list of evaluated tensors as numpy matrices.
train_step(datadict, eval_tensors=[], update=True)[source]

Performs a training step of gradient descent with given optimization strategy.

Parameters:
  • datadict – A dictionary of names (str) matching names in ph_dict to numpy matrices for this mini-batch.
  • eval_tensors – (list of Tensors) Tensors to evaluate along with train_op.
  • update – (boolean) Whether to perform a gradient update this train step
Returns:

A list of numpy arrays for eval_tensors. First element is None.
graph_training_utils.get_feed_dict(datadict, ph_dict, train=1, debug=False)[source]

Function for pairing placeholders of a tensorflow computational graph with numpy arrays.

Parameters:
  • datadict – A dictionary with keys matching keys in ph_dict, and values are numpy arrays.
  • ph_dict – A dictionary where the keys match keys in datadict and values are placeholder tensors.
  • train – {1,0}. Different values get fed to placeholders for dropout probability, and batch norm statistics depending on if model is training or evaluating.
  • debug – (boolean) Whether or not to print dimensions of contents of placeholderdict, and datadict.
Returns:

A feed dictionary with keys of placeholder tensors and values of numpy matrices.