Category Archives: Machine Learning

Tensorflow 1.x and 2.x Saving Error Using a `tf.Tensor` as a Python `bool` is not allowed.

Ever tried to save a Tensorflow model with tf.compat.v1.saved_model.simple_save or a similar TF saving method function?

Ever Encounterd in TF 1,X
TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if t is not None: instead of if t: to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
or in TF 2.X
OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

Error
DONT BE FOOLED! This error is not what it seems. At least for me…

This was my save method that was causing the error


token_tensor = tf.ones((input_len,batch_size), "int32", "token_tensor")
segment_tensor = tf.ones((input_len,batch_size), "int32", "segment_tensor")
mask_tensor = tf.ones((input_len,batch_size), "float32", "mask_tensor")
seq_out = model.get_sequence_output()

with tf.compat.v1.Session() as sess:
tf.compat.v1.saved_model.simple_save(
sess,
export_dir,
inputs={'input': token_tensor, 'segment' : segment_tensor, 'mask' : mask_tensor},
outputs=seq_out,
legacy_init_op=init_op
)

See the error? Its very minor…
The problem was, the output tensor IS NOT INSIDE OF A DICT !
Duuuuh! Isn’t that obvious to infer from the

Looking at the source code of the save function is what actually made me see the issue!
simple_save.py

So here is the fix, just define a dict for your input or output tensors!

token_tensor = tf.ones((input_len,batch_size), "int32", "token_tensor")
segment_tensor = tf.ones((input_len,batch_size), "int32", "segment_tensor")
mask_tensor = tf.ones((input_len,batch_size), "float32", "mask_tensor")
seq_out = model.get_sequence_output()

with tf.compat.v1.Session() as sess:
tf.compat.v1.saved_model.simple_save(
sess,
export_dir,
inputs={'input': token_tensor, 'segment' : segment_tensor, 'mask' : mask_tensor},
outputs={"out": seq_out},
legacy_init_op=init_op
)

Happy TensorFlow hacking!

This is the full Stack trace in TF 1.X
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
45 outputs= seq_out, #{'output': mask_tensor, 'norms': mask_tensor},
46 #outputs={'word_emb': model_wordembedding_output, 'sentence_emb': model_sentence_embedding_output},
---> 47 legacy_init_op=init_op
48 )
49 # print('saving done')

/home/loan/venv/XLNET_jupyter_venv/lib/python2.7/site-packages/tensorflow/python/util/deprecation.pyc in new_func(*args, **kwargs)
322 'in a future version' if date is None else ('after %s' % date),
323 instructions)
--> 324 return func(*args, **kwargs)
325 return tf_decorator.make_decorator(
326 func, new_func, 'deprecated',

/home/loan/venv/XLNET_jupyter_venv/lib/python2.7/site-packages/tensorflow/python/saved_model/simple_save.pyc in simple_save(session, export_dir, inputs, outputs, legacy_init_op)
79 signature_def_map = {
80 signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
---> 81 signature_def_utils.predict_signature_def(inputs, outputs)
82 }
83 b = builder.SavedModelBuilder(export_dir)

/home/loan/venv/XLNET_jupyter_venv/lib/python2.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.pyc in predict_signature_def(inputs, outputs)
195 if inputs is None or not inputs:
196 raise ValueError('Prediction inputs cannot be None or empty.')
--> 197 if outputs is None or not outputs:
198 raise ValueError('Prediction outputs cannot be None or empty.')
199

/home/loan/venv/XLNET_jupyter_venv/lib/python2.7/site-packages/tensorflow/python/framework/ops.pyc in __nonzero__(self)
702 TypeError.
703 """
--> 704 raise TypeError("Using a tf.Tensor as a Python bool is not allowed. "
705 "Use if t is not None: instead of if t: to test if a "
706 "tensor is defined, and use TensorFlow ops such as "

TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if t is not None: instead of if t: to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

And Tensorflow 2.x

---------------------------------------------------------------------------
OperatorNotAllowedInGraphError Traceback (most recent call last)
in
86 inputs=bert_inputs,
87 outputs=table_tensor,
---> 88 legacy_init_op=init_op
89 )
90

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py in new_func(*args, **kwargs)
322 'in a future version' if date is None else ('after %s' % date),
323 instructions)
--> 324 return func(*args, **kwargs)
325 return tf_decorator.make_decorator(
326 func, new_func, 'deprecated',

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/saved_model/simple_save.py in simple_save(session, export_dir, inputs, outputs, legacy_init_op)
79 signature_def_map = {
80 signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
---> 81 signature_def_utils.predict_signature_def(inputs, outputs)
82 }
83 b = builder.SavedModelBuilder(export_dir)

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py in predict_signature_def(inputs, outputs)
195 if inputs is None or not inputs:
196 raise ValueError('Prediction inputs cannot be None or empty.')
--> 197 if outputs is None or not outputs:
198 raise ValueError('Prediction outputs cannot be None or empty.')
199

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in __bool__(self)
755 TypeError.
756 """
--> 757 self._disallow_bool_casting()
758
759 def __nonzero__(self):

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in _disallow_bool_casting(self)
524 else:
525 # Default: V1-style Graph execution.
--> 526 self._disallow_in_graph_mode("using a tf.Tensor as a Python bool")
527
528 def _disallow_iteration(self):

~/venv/XLNET_py3_venv/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in _disallow_in_graph_mode(self, task)
513 raise errors.OperatorNotAllowedInGraphError(
514 "{} is not allowed in Graph execution. Use Eager execution or decorate"
--> 515 " this function with @tf.function.".format(task))
516
517 def _disallow_bool_casting(self):

OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

Convolution Operators

Convolution Operators

Overview of  a few convolution operators

Sobel Operator for Edge Detection

The Sobel Operator calculates the pixels first order derivatives in the X and Y direction.

If the is little derivative, there is no unstructured area

If the derivative high in one direction, we have an edge in that direction

If the derivative high in two directions  , we have a corner / point of interest

 Gaussian Smoothing

Replace each pixel by it’s local average

.It is used for scale Space

The Laplace  Operator for

The differnce compard to the Sobel operator is, that it uses the second order derrivative.

This makes the Laplace operator very sensitive to noisy.

Edges are, where the second derivative is crossing over to 0 ( Zweite ableitung = 0 Hochpunkt!)

Laplacian of Gaussian Filter :

  1.  First smooth with gaussian filter.

  2. Then find zero crossings with the Laplacian filter.

  3. Can also combine one into one LOG  convolution

Double of Gaussian

LoG does not have to be calculated, it can be also approximated by calculating the difference between two Gaussian Filters at different scales . DoG

SIFT Detector

  1. Multiple DoG filters are applied to the image at different scales
  2. The resulting images are stacked on top each other to create a 3D volume
  3. Points that are local extrema inside of the 3D voluma, are considerd points of interest.  
  4. Remove bad points, like candidates in smooth regions  or directly on top of edges

SIFT Discriptor

  1. For a detected point of interest, choose 16×16 region around the point
  2. Compute each gradient for each pixel
  3. Subdivide into 16 * 4×4 groups
  4. Compute orientation histogram
  5. Glue the histograms together to get a 128 element feature vector

Characteristics

  • SIFT is very resillient to changes (invariant)   to constant intensity changes, as itt is based on gradients
  • Very invariant to contreast changes  , as the histogram binning provides normalization
  • invarient to small deformations  

Scale invariance is caused by the SIFT detector