Problem loading a simple saved model

[wurzel400]

I have problems when running the following mini example with Tensorflow V2.17 (similar problems also with V2.16):

import tensorflow as tf
import numpy as np

data_x=np.random.random((64,10))
data_y=np.random.randint(2,size=len(data_x),dtype=np.uint8)

model = tf.keras.Sequential([tf.keras.layers.InputLayer(input_shape=(10,)),
                             tf.keras.layers.Dense(5,activation=tf.keras.layers.ReLU()),
                             tf.keras.layers.Dense(1,activation=tf.keras.layers.ReLU())])
model.build()
model.compile(optimizer=tf.keras.optimizers.AdamW(), loss=tf.keras.losses.MeanAbsoluteError())
model.summary()

splitpoint=int(len(data_x)*0.9)
train_x=data_x[0:splitpoint]
train_y=data_y[0:splitpoint:]
val_x=data_x[splitpoint:]
val_y=data_y[splitpoint:]
history = model.fit(x=train_x, y=train_y, validation_data=(val_x,val_y), batch_size=16, epochs=3)

tf.keras.models.save_model(model, filepath="mymodel.keras")
loaded_model=tf.keras.models.load_model("mymodel.keras")
loaded_model.summary()

I see the output of the first summary(), and the model gets trained and saved. But loading fails with the following error message on load_model() in the penultimate line:

Traceback (most recent call last):
  File "C:\MyData\tmp\tmp_env_pw_det\Lib\site-packages\keras\src\ops\operation.py", line 234, in from_config
    return cls(**config)
           ^^^^^^^^^^^^^
  File "C:\MyData\tmp\tmp_env_pw_det\Lib\site-packages\keras\src\layers\core\dense.py", line 89, in __init__
    self.activation = activations.get(activation)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\MyData\tmp\tmp_env_pw_det\Lib\site-packages\keras\src\activations\__init__.py", line 104, in get
    raise ValueError(
ValueError: Could not interpret activation function identifier: {'module': 'keras.layers', 'class_name': 'ReLU', 'config': {'name': 're_lu', 'trainable': True, 'dtype': {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None, 'shared_object_id': 2116176554640}, 'max_value': None, 'negative_slope': 0.0, 'threshold': 0.0}, 'registered_name': None, 'build_config': {'input_shape': [None, 5]}}

During handling of the above exception, another exception occurred: [...]

With Tensorflow V2.15 (and prior), everything ran fine. Am I doing something wrong or is this a bug?

[ghsanti]

Hi @wurzel400, does any of the following work?

• Replacing activation=tf.keras.layers.ReLU() by activation=tf.keras.activations.relu.

• Use ReLU layer instead of a Dense layer.

Test (using Keras 3) can be found here.

[wurzel400]

@ghsanti Both of your suggestions worked, thanks a lot!

Nevertheless, I find it somehow strange that

1. My approach worked before (i.e., with TF version < 2.16)
2. With TF>=2.16, training worked, but inference failed. If what I did is not the intended way, shouldn´t you be notified already when training starts? Otherwise, you may notice only after weeks of training that your stored model is not loadable.

[ghsanti]

Both of your suggestions worked, thanks a lot!

You are welcome :-)

1. Compatibility can be an issue indeed; if you can, try targeting the latest stable versions (see next item.)

2. With respect to:

   With TF>=2.16, training worked, but inference failed.

   • If the code works with keras~=3.5.0 and tensorflow~=2.17.0 I wouldn't worry, since that means it was (likely) fixed between the versions!
   • If it does not, please add a Colab link that reproduces the error and someone will take a look.

3. If you can't target the those versions for some reason, feel free to expand.