It seems CreatePredictionEngine overrides inputSchemaDefinition by calling into DataViewConstructionUtils.GetSchemaDefinition. This is problematic if some of the features of input schema is determined at runtime, example, length of feature vector.
ITransformer trainedModel;
using (var stream = new FileStream(modelFileFullPath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
var trainedModel = mlContext.Model.Load(stream, out var inputSchema);
var = mlContext.Model.CreatePredictionEngine<TInput, TOutput>(trainedModel, inputSchemaDefinition: inputSchema);
//Throws an exception features<vector<single>> does not match features<vector<single, 432>> because CreatePredictionEngine returns transformer.CreatePredictionEngine<TSrc, TDst>(_env, false, DataViewConstructionUtils.GetSchemaDefinition<TSrc>(_env, inputSchema)); and `DataViewConstructionUtils.GetSchemaDefinition` overrides inputSchema.
}
//Below workaround that overrides the override!
using (var stream = new FileStream(modelFileFullPath", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var trainedModel = mlContext.Model.Load(stream, out var inputSchema);
var outputSchemaDefinition = SchemaDefinition.Create(typeof(TInput));
outputSchemaDefinition["Features"].ColumnType = new VectorDataViewType(NumberDataViewType.Single, Convert.ToInt32(432));
var p = mlContext.Model.CreatePredictionEngine<TInput, TOutput>(trainedModel, inputSchemaDefinition: outputSchemaDefinition);
}
CC: @TomFinley
It seems
CreatePredictionEngineoverridesinputSchemaDefinitionby calling intoDataViewConstructionUtils.GetSchemaDefinition. This is problematic if some of the features of input schema is determined at runtime, example, length of feature vector.CC: @TomFinley