Classification of nuts step 3

Breeze can use a wide range of generic and specific Machine learning algorithms. Breeze can also import models created in other Machine learning frameworks which can export the models to ONNX format.

Before doing this tutorial, first, do the tutorial: Classification of nuts step 2

In this tutorial, you will create a Representative spectrum to expand on the existing data. Train a machine-learning algorithm to perform classification using the Auto evaluation mode to have Breeze select and train the most suitable algorithm. Inspect and fine-tune the model, and finally have a ready-to-run real-time classification workflow.

Steps included in the tutorial

• Create representative spectrum

• Create a machine learning classification model

• Real time predictions

Create representative spectrum

Open the study for the “Classification of Nuts step 2” tutorial that you have previously done. In Record on the Study level enter the Analysis Tree tab and add a new node after the segmentation object node by pressing the Object symbol and then the plus sign.

Select the Segmentation tab in the “Add node” window and select the “Representative spectrum” method.

Set Method as “Random (Gaussian)” for a random distribution with a Gaussian (normal) distribution.

Set “Number” to 50 for a total of 50 pixels to be included from each object using the random Gaussian distribution. Set “Unique” to “True” so no pixels are repeated even if all pixels in the object have already been selected.

Press Apply changes to apply the new segmentation step

Go to the Table tab and select the new “Representative spectrum” segmentation in the drop-down menu under the table. In the image on the left of the screen, you see that 50 pixels have been selected from each object.

Go to the Model view.

Create a Machine Learning model

Press “Add” and create a new Classification model selecting the Method “Machine Learning”.

Use suggested default model name and press “Ok”.

Select the newly created Segmentation “Representative spectrum” and “Nut type” as the variables to classify and click Next.

Click the “Next” button.

Here is now a list of all pixels that have been segmented out. We will use some of them as a training set and some as the test set. Scroll the list to the right so you can see the “Group” column. Select all the rows that are in the Group “Test” (select the first Test sample, hold down shift on your keyboard, and then scroll down to the last Test sample in the list and then click on it to select all samples in the Test group)

Press the “Test” button.

If you scroll to the left again you will see in the “Type” column that these samples are now selected as “Test” and the rest are “Train”.

Click the “Next” button to inspect which wavelengths will be used in the model.

Check the “Show SNR“ checkbox to view the Signal-to-noise ratio. In the example below we order the SNR values in ascending order (least Signal-to-noise ratio) and Exclude the wavelengths with SNR values below 1.6.

We then check the “SNV (Standard Normal Variate correction)“ (see Pretreatments | SNV-(Standard-Normal-Variate-correction) for more information).

Click the “Next” button to accept the selections.

In the “Model” step of the wizard, set the training Time to 120 (seconds) and the Algorithm to Auto (default). Then click “Train” to have Breeze perform an automatic evaluation of the training data using cross-validation to find the best algorithm.

The table is sorted in descending order of performance. The algorithm on the top has the highest accuracy and will be used. Click “Finish” to accept the result and create a model of the best-performing algorithm

Click the “Play” button.

Real-time predictions

Click the “Add” button.

Select the “Record data” tab and make sure the “Test” group is selected, check the “Include categories and properties from Record” option.

In the “Table” tab you may notice several columns, the first two are the actual (correct) classes that we had added for each sample. In this machine learning model we only used the “Nut type” column for the training. Then you have one column that is the results of the prediction when using the Machine Learning classification model created above (press Apply changes if you don’t seen any data in these columns).

You can right-click on the column header and delete the “Nut or Shell” column as we are not going to look at this classification now.

Inspect the results between the predicted Nut type and actual. Right-click on the prediction column “Nut type” and click “Show classification summary”.

The summary will indicate if there are any misclassifications of the objects. In this example, one Hazelnut is misclassified as an Almond (your results might vary slightly).

When inspecting the table we can see that for the misclassified nut (i.e. Hazelnut misclassified as Almond) the majority of the pixels are classified correctly, but the object classification is incorrect (this may not be the case in your model). The misclassification of the object in this case is due to the fact that the prediction is done at an object-level using the object's average spectrum, while the training data we used where based on 50 pixels from each object.

To remedy this we can change to object classification method, switching to the Analyse Tree tab and selecting the classification node. Change the model “Classification type” from “Object average spectrum” to “Pixels class majority”. This will classify the object based on the majority class of the pixels.

You can keep the default “Weights” (2;5;10)

Which are the weight of the pixels as such:

• 2 - closest to the edge of the object

• 5 - second closest to the edge of the object

• 10 - the center of the object.

The default weight implies the center of the object is most representative of the spectrum and closer to the edge is less so (entering the value 1 would make app pixels equally important).

Click “Apply Changes”.

Now let us inspect the Table to see if the samples are correctly classified

Right click on the header for Nut type and the press “Show classification summary”

Your model should hopefully also receive a perfect object classification.

The model classification is now ready for real-time predictions. To test this, click on Recorder and the same images used in training the model and in the test set will be evaluated in a real-time scenario.

Select the existing groups or make a new group. Uncheck the “Save image measurements and calculated descriptors” option and select “Parallel” Measurement segmentation.

Click on “Play”.