Training ONNX model using external python script file
An external python script file may be used to train a model outside of Breeze. The result is then imported into Breeze and may be used as any model trained inside of Breeze.
TIP On Windows you can install python and dependencies to run the example files as part of the installation of Breeze:

Execution of script file
[py|python] SCRIPT_FILE TRAINING_CSV TEST_CSV ONNX_DESTINATION TRAINING_INFORMATION_FILE
On Windows py
command is used and on macOS and Linux python
is used.
The standard output is shown in a log window in Breeze during the execution of the script.

Windows file selection dialog allows you to selected which python interpreter to use
Requirements
A valid default python environment (with all relevant dependencies installed)
Consumption of 4 positional arguments (see below)
Creation of 2 files
ONNX model file
With valid opsets for all relevant dependencies
JSON file in specific format (see below)
Positional arguments
TRAINING_CSV
- The absolute path to a csv file containing the label and wavelength information for the training data set.TEST_CSV
- The absolute path to a csv file containing the label and wavelength information for the test data set.ONNX_DESTINATION
- The .onnx file which the script file should have created after execution.Will be in Breeze Temp folder
Does not exist when the script is executed and must be created
TRAINING_INFORMATION_FILE
- Training results in the format belowWill be in Breeze Temp folder
Does not exist when the script is executed and must be created
See format below
TEST_CSV
file will exists but may be empty
Training data
The training and test data will be in csv (;
separated) format. Below is an example of a SNV treated spectrum. Starting with the true class label value of 1
then the treated wavelength data.
1;1.2151460647583008;1.9284714460372925;....;N
The format will be the same for the test data set.
Example script file
Below is a simple neural network defined using TensorFlow and Keras (see attachments for complete script file).
The script requires python version 3.7-3.10
Additional example scripts can be found in the Breeze Runtime install directory: Breeze\Runtime\ExampleCode\TrainPythonModel
The positional arguments are read into variables using:
TRAIN_DATA = sys.argv[-4]
TEST_DATA = sys.argv[-3]
ONNX_DESTINATION = sys.argv[-2]
TRAINING_INFORMATION_FILE = sys.argv[-1]
The model definition must follow the import format (see Import of ONNX model )
inputs = keras.Input(shape=(wls,), name="Features")
x = inputs
x = keras.layers.Dense(units=6, activation=tf.nn.tanh)(x)
x = keras.layers.Dense(units=6, activation=tf.nn.tanh)(x)
x = keras.layers.Dense(units=6, activation=tf.nn.tanh)(x)
y1 = keras.layers.Dense(number_of_classes, activation=tf.nn.softmax, name="Score.output")(x)
# Custom layer for argmax value to Breeze class label
y2input = Lookup(classes)(y1)
y2 = keras.layers.Lambda(lambda xx: xx, name='PredictedLabel.output')(y2input)
model = keras.Model(inputs, [y1, y2])
When the model is trained the conversation to ONNX format should be done in the script.
tf2onnx.convert.from_keras(
model,
opset=15,
output_path=ONNX_DESTINATION
)
And the training information written to the destination JSON file.
modelTrainingInfo = {
"Accuracy": str(score_training_data[-1]),
"AccuracyTest": str(score_test_data[-1]) if score_test_data is not None else "-1",
"RuntimeInSeconds": str(end_time - start_time),
"AlgorithmName": "External",
"DisplayName": "Neural Network",
}
wrapper = {
"Results": [modelTrainingInfo]
}
with open(TRAINING_INFORMATION_FILE, "w") as f:
json.dump(wrapper, f)
Expected result
A model evaluation table with the results from the TRAINING_INFORMATION_FILE
file. The ONNX model file will be imported and the test set applied (or training set if no test set) after the training outside of Breeze.

Model training evaluation table in Model Training Wizard
TRAINING_INFORMATION_FILE
format
{
"Results": [
{
"Accuracy": "0.9798425436019897", # Training result
"AccuracyTest": "-1", # Result from test set (if applicable else "-1")
"CrossValidationResults": "-1" # Result from cross validation (if applicable else "-1")
"RuntimeInSeconds": "11.319475650787354", # Runtime in seconds
"DisplayName": "Neural Network" # Any non-empty string - to be displayed in table
}
]
}
Any number of Results
may be added to the JSON file. The first result will be marked as used in Breeze for the purpose of presentation.