ValueError: Shapes (8, 1, 1, 12) and (8, 16, 16, 12) are incompatible

[aleguillou1]

Hello, I'm posting here because I'm having a problem training an FCN with fccn_model.py. I'm trying to train a model to classify into 12 land use classes with a SPOT6 image in 4 spectral bands.

I followed the tutorial, extracted my patches from the sample selection then created my patches with the patch extraction and I even created in TFrecord afterwards. But when I try to train the model by specifying in the code (N_CLASSES=12), I get the following error that I can't solve:

File "/opt/otbtf/lib/python3/dist-packages/keras/engine/training.py", line 1284, in train_function  *
    return step_function(self, iterator)
File "/opt/otbtf/lib/python3/dist-packages/keras/engine/training.py", line 1268, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/opt/otbtf/lib/python3/dist-packages/keras/engine/training.py", line 1249, in run_step  **
    outputs = model.train_step(data)
File "/opt/otbtf/lib/python3/dist-packages/keras/engine/training.py", line 1051, in train_step
    loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/opt/otbtf/lib/python3/dist-packages/keras/engine/training.py", line 1109, in compute_loss
    return self.compiled_loss(
File "/opt/otbtf/lib/python3/dist-packages/keras/engine/compile_utils.py", line 265, in __call__
    loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "/opt/otbtf/lib/python3/dist-packages/keras/losses.py", line 142, in __call__
    losses = call_fn(y_true, y_pred)
File "/opt/otbtf/lib/python3/dist-packages/keras/losses.py", line 268, in call  **
    return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "/opt/otbtf/lib/python3/dist-packages/keras/losses.py", line 1984, in categorical_crossentropy
    return backend.categorical_crossentropy(
File "/opt/otbtf/lib/python3/dist-packages/keras/backend.py", line 5559, in categorical_crossentropy
    target.shape.assert_is_compatible_with(output.shape)

ValueError: Shapes (8, 1, 1, 12) and (8, 16, 16, 12) are incompatible

This may be due to the labels being 1x1 in input with PatchExtraction, but I can't solve the problem.
The model understands that in output it needs 12 outputs because I have the table showing the convolutions before the error that happens as soon as it tries to train from the 1st Epoch: predictions_softmax_tensor_crop96 (Activation) (None, None, None, 12) 0 ['tf.operators.getitem_3[0][0]'] (the last output) , it also finds the input shape very well: Original shape for input input_xs: [16, 16, 4]
2023-11-30 10:17:06 INFO New shape for input input_xs: [None, None, 4]
2023-11-30 10:17:06 INF Model inputs: {'input_xs': <KerasTensor: shape=(None, None, None, 4) dtype=float32 (created by layer 'input_xs')>}
2023-11-30 10:17:06 INFO Normalized model inputs: {'input_xs': <KerasTensor: shape=(None, None, None, 4) dtype=float32 (created by layer 'tf.math.multiply')>}
2023-11-30 10:17:06 INFO Model outputs: {'predictions': <KerasTensor: shape=(None, None, None, 12) dtype=float32 (created by layer 'predictions_softmax_tensor')>}

Thanks in advance! :)

[remicres]

Hi @Noxvie, if you want to extract labels greater than 1x1, you must specify another source with the correct patch size (Looks like it should be 16x16 in your case)

You can take a look here how to achieve that.

Hope that helps

[aleguillou1]

Hello Rémi,

First of all, thank you very much for the speed of your reply. I took a little time to reply so that I could have a good look at the forum page that you sent, but unfortunately I wasn't able to solve my problem, but it's probably my fault because everything wasn't very clear to me.

Maybe I didn't express myself well in my question. I'm trying to carry out a classic classification of 12 land use classes from a single satellite image (SPOT6) with 4 spectral bands. To do this I've carried out the same process as at the time of otbtf on TFV1, so I've done PatchesExtraction for a 16x16 patch with a 1x1 label from the Sample Selection tool, etc. Maybe that's what's blocking me. I saw that the code was intended more for semantic segmentation, in which case it would be preferable for me to simply create a CNN from OTBTFV1 as it is more suitable than the FCN?

For example, by setting OTB_TF_NSOURCES to 2, the processing chain (SampleExtraction) asks me for two satellite image inputs, whereas I only have one patch file (and therefore one label file) taken from a single satellite image to train my model. I've also seen that it's not mandatory to use the label code, but to train your model in Tfrecords (but also from patchextraction only) you have to give the labels as input with the patches, so I don't understand the point of not specifying labels.

Best regards,
Adrien

[remicres]

Let's talk about PatchesExtraction. Setting OTB_TF_NSOURCES=2 is required to extract 2 sets of patches. In the specific case of semantic segmentation, you have typically a 16x16 input image ("source1.out") and another 16x16 label image ("source2.out"). And you don't use "outlabels".
Now in the case of a sparsely annotated ground truth, you might consider to train a network from one 16x16 input image ("source1.out") and a label value of size 1x1 ("outlabels"). In this case, you need only 1 source.

Now let's talk about your model training.

• 16x16 input image --> 16x16 labels: any U-Net-like model
• 16x16 input image --> 1x1 labels: FCNN model (e.g. a model composed of 5 conv2d of 4x4 with stride 1)

Hope that helps

[aleguillou1]

Hello Rémi,

Sorry for taking so long to reply. I had other things to do and I also tried to take the time to understand my mistakes. Indeed it's much clearer, the architecture of my CNN was not adapted to the sizes of the patches and my labels that I had made.

Thank you very much, have a nice day!