How to run onnx model with batch/sequence

[EmileVinsonneau]

Description

My usecase is to implement runtime in .Net/C#, on windows, to use onnx model like this :

My issue is simple : I am not able to predict anything, I got nullreference exception.
I don't know how implement tensors.
And I am lock, I have no idea and I don't know how to search.

how I did

My code is splited into 3 parts :

• initialization
• tensor class
• prediction

        this.mlContext = new MLContext();

        var inputColumns = new string[] { "input", "sr", "h", "c", };
        var outputColumns = new string[] { "output", "hn", "cn", };

        this.predictionPipeline = mlContext.Transforms.ApplyOnnxModel(
            outputColumnNames: outputColumns,
            inputColumnNames: inputColumns,
            modelFile: this.pathToOnnxModelFile);
        var emptyDv = mlContext.Data.LoadFromEnumerable(new SileroInput[] { });
        var model = predictionPipeline.Fit(emptyDv);
        this.predictionEngine = mlContext.Model.CreatePredictionEngine<SileroInput, SileroOutput>(model);

• input tensor class and output tensor class

public class SileroInput
{
    public readonly int batch = 1;
    public readonly int sequence = 1024;

    [ColumnName("input")] public float[,] Audio { get; set; }

    [ColumnName("sr")] public long SampleRate { get; set; }

    [ColumnName("h")] public float[,,] H { get; set; }

    [ColumnName("c")] public float[,,] C { get; set; }

    public SileroInput()
    {
        this.Audio = new float[this.batch, this.sequence];
        this.H = new float[2, batch, 64];
        this.C = new float[2, batch, 64];
    }
}

public class SileroOutput : SileroInput
{
    [ColumnName("output")]  public float[,] Probability { get; set; }
    
    [ColumnName("hn")] public float[,,] HN { get; set; }

    [ColumnName("cn")] public float[,,] CN { get; set; }

    public SileroOutput()
    {
        this.Probability = new float[this.batch, 1];
        this.HN = new float[2, this.batch, 64];
        this.CN = new float[2, this.batch, 64];
    }
}

while()
{
  var audio = getAudio(); // piece of audio
  this.input = audio;
  var prediction = this.predictionEngine.Predict(input); // here, I have null ref

  this.input.H = prediction.HN;
  this.input.C = prediction.CN;

   isActive = prediction.Probability[0,0] > .5D;
}

I feel the way I declare my tensors are wrong, but exception do not help me.
I write here the simpliest way in my mind, I try different way to implement tensor but ... it's critical failed.

Any help ...

[EmileVinsonneau]

After reading this dotnet/machinelearning#6066

I change my input and output to have only dimensionnal array.

public class SileroInput
{
    protected const int batch = 1; 

    [ColumnName("input")]
    public float[] Audio { get; set; }

    [ColumnName("sr")]
    public long SampleRate { get; set; }
    
    [ColumnName("h")]
    public float[] H { get; set; } = new float[2 * batch * 64];
    
    [ColumnName("c")]
    public float[] C { get; set; }= new float[2 * batch * 64];
}

public class SileroOutput : SileroInput
{
    [ColumnName("output")]
    public Single[] Probability { get; set; } = new float[batch * 1];
    
    [ColumnName("hn")]
    public Single[] HN { get; set; } = new float[2 * batch * 64];

    [ColumnName("cn")]
    public Single[] CN { get; set; }= new float[2 * batch * 64];
}