Intro to Breeze: Classification of nuts step 1

A Breeze installer wizard will appear. Follow the steps in the wizard to complete the installation.

Press Next

You will be asked to select a camera supplier.

Select the camera you plan to use with Breeze for data acquisition.

For this tutorial select we can select No external camera.

Press Next

After starting Breeze for the first time a pop up will appear with links to our Help site with tutorials.

Press OK to close it.

When the help pop up is not visible, you can find the tutorial on the Breeze Home view.

Click on Tutorial to start.

Select “Nuts Classification” in the “Name” drop-down menu.

Press OK to start downloading the image data.

Always choose a project to work on. That is done here i the home view of Breeze.

There are three different way to open the project…

1. Double click on project

2. Click once and choose Analyzer

3. Right click and choose Analyzer

The image data in this Project is organized into two Groups (folders) called “Train” and “Test” and includes eight training images (with either nuts or shells) and one test image.

You can click on a table row to see the preview image (pseudo-RGB) for each image.

Double click on the “Train” group to open it.

In the menu on the left side, you see all the individual images (called Measurements), in this group.

Click on the Pixel Explore tab to look closer at an individual image and its spectral data. In this example we have selected the first image “Almond”

Collapse the left menu using the small arrow to see expand the “Pixel Explore” view.

By hovering the mouse over the image you can see the spectrum for the pixel at the position of the cursor.

Hold down left mouse button and make a selection of an area of the image. The average spectrum for this area will be shown in the spectrum plot.

You can compare different areas by making additional selections.

In the menu on the right side are options for different Selection and Zoom tools. You can also zoom by using the scroll function on your mouse.

The spectral plot has 3 vertical lines showing the spectral bands used to generate the pseudo-RGB image. Can can modify this by sliding the lines using your mouse.

Now let’s do a quick analysis of the spectral variation in the image, by clicking on the + in Model (PCA)

A PCA model has been created based on all pixels in the image. Each point in the “Variance scatter” plot corresponds to a pixel in the image. The points in the scatter plot are clustered based on spectral similarity. The color of the points in the scatter plot is based on density (i.e. red = many points close to each other).

The “Max variance image” is colored by the variation in the 1st component of the PCA model (the X-axis in the scatter plot, t1), and visualizes the biggest spectral variation in the image. In this case, this is the difference between the sample (blue) and the background (orange).

Do a selection of a cluster of points in the scatter plot. These pixels are automatically highlighted in the image.

Go back to the project level by clicking on the Back button in the upper left corner.

Select “Nuts-Classification” in the menu on the right side panel to select the project. This will enable you to see all images from both the “Train” and “Test” groups in Table.

We will now add a new class variable to your training data set so we can define the true values. This is done by creating a new column in the table.

Click “Add variable or Id” available at the bottom of the page.

Select type “Category (Classification variable)” and write the name “Nut or shell” and press Add.

A column for “Nut or shell” has now been added to the Table

Categorize all nuts, by pressing the Ctrl-key and select with the mouse cursor “Almonds”, “Hazelnut”, “Pecans” and “Walnuts” samples.

Right-click on one of the selected rows in the “Nut or Shell” column and write the class name “Nut” in the “New Class” field and press select “Add”.

Do the same thing again for the shell samples and categorize them with the new class “Shell”.

Your table should look like this now.

You do not need to set a class to the Test image.

Press “Add Sample model” button at the bottom of the screen:

Choose a name for the sample model (or use the default name)

press “OK”

A model wizard will appear.

In the first step - select the Measurements (images), that you will use in the model.

By default, all measurements are included, which is ok.

Press “Next”

Step two - Select Wavelengths (spectral bands) to use in the model.

On the left side you see a table of all spectral bands. On the right you have a spectral plot showing the average spectrum of each image. By default, all wavelengths are included as shown by being highlighted in green.

You can also see the Pre-treatment steps that can be applied to the data. By default “SNV (Standard Normal Variate correction)“ is selected (see Pretreatments | SNV-(Standard-Normal-Variate-correction) for more information).

Press “Next”

Step 3 - Model, select the pixels to use to train the sample model.

A mosaic has been created of all images and a PCA model has been created from all the pixels in this mosaic.

In the scatter plot, select all pixels in the cluster on the right side (use the pixel density coloring red, yellow, green, and light blue as help).

The corresponding pixels are then selected in the image on the right side and should correspond to all nut and shell pixels.

Press the “Include Only” button in the menu:

The plots are now updated and will contain mostly the nuts pixels.

To clean up the nuts pixels even more you can remove the pixels bordering each sample object.
Press the “Select - Border pixels” button.

In pop up - Use the default of 1 border pixel and press “OK”.

The border pixels have now been selected.

Exclude the border pixels by pressing the “Exclude” button

Press Next

Step 4 - Critical Distance, set the Critical Distance threshold.

This is the distance to the sample model and will be used to determine if pixels are sample or not. The histogram is showing the distance to the model for all pixels in the images. Pixels on the left side of the red vertical line (critical distance) are inside the threshold.

Drag the red line to the right to move the threshold. As you can see from image to the right, more pixels are included when doing this.

The aim is to find a level where all nut pixels are included but not pixels from the background.

As a general recommendation, you can drag the red line to the bottom of the “valley” between the sample and the background bars in the histogram as shown below.

Press “Next”

Step 5 - Approve, the “Minimum area size” is used to automatically exclude smaller unwanted objects (for example dust or dirt).

Breeze calculates a suggested minimum area size for your data. In this example, any objects under 300 pixels will be excluded from the image. Depending on how you did the pixel selection in the previous step this value might vary. A value around 300 should be OK.

Press “Finish” to create the sample model and apply this to all images in the project.

In the “Table” for the project Nuts_Classification, you can now see all the unique sample objects in the images after the sample model has been applied and the background pixels removed.

Collapse the left structure to view the results better.

Click on a sample in the “Nut or shell” column in the table to color all objects in the preview image based on the class.

You can also click on the objects in the preview image to see where they are in the table.

Press the “Explore” tab. A PCA model has been created based on the average spectrum for each sample. Each point in the scatter plot corresponds to a sample and the points are clustered based on spectral similarity. Select one or several points to see their average spectrum.

In the menu on the right click on the category name “Nut or shell” to color the scatter plot and preview the image based on the different classes (the red dots are the “Mix” samples where we had not entered any class).

(you can close the preview image with the arrow to make more space for the plots and graphs)..

To create another model, go to the Model view by pressing the button on the left side of the screen.

In the menu on the left, you can see the Sample model that you created before.

To make an additional model Press the “Add” button in lower left corner.

On the window that appears, press the “Classification” tab, and then select “PLS-DA”.

Write a name for the model (or use the default name)

press “OK”.

Step 1 - Variables

The Segmentation using the Sample model will be used.

The “Nut or shell” variable has been selected and will be used to build the model.

press “Next”.

Step 2 - Samples

Select the samples that you want to include in the model.

By default, the measurements from the “Train” group have been included since they have entered class information.

The measurement “Mix” in the “Test” group has been excluded since there was no class information entered. This is OK.

Press “Next”

Step 3 - Wavelength

By default, all wavelength bands are included. The graph on the right is showing the average spectrum for each sample.

By default “SNV (Standard Normal Variate correction)“ is used. (see Pretreatments | SNV-(Standard-Normal-Variate-correction) for more information).

Press next

Step 4 - Model

A PLS-DA classification model has now been calculated.

FYI - The results on your screen might vary slightly from these result seen here due to how you set the Critical Distance threshold in the previous segmentation.

Step 5 - Evaluate how good the model is.

The “Nut or shell” vs. “Ycalc.

Nut or shell” is showing how well the model can separate the two classes.

The “Variable overview” is showing the R2 and Q2 for each class.

Everything looks OK.

Press “Finish” to complete the model.

The Classification PLS-DA model has now been saved as you can see in the menu on the left.

With this model selected press the “Classification” tab to see how many samples were correctly classified.

Each row is showing the correct class for the samples, and the columns are showing the classes that these samples are classified into by the model.

In this example, all the samples are correctly classified.

Close Model explorer

Press the Analyse Tree tab at the top of the screen.

Here you can make your own data processing workflow to automate the analysis of the images in your project.

The sample model we developed before is already in the Analysis Tree as the first processing step of your data (Measurements).

Press the “+“ symbol after “Object”

A pop-up will appear where you can select the data processing function you want to apply.

Press Descriptor and then select Classification of Categories

The “Nut or shell” PLS-DA model has now been added to the Analyse Tree.

When we apply this to our data, first, the “Measurement” (image) is analyzed by your sample model (“Sample - Nuts_Classification”) to find the sample “Object”.

For this object, it then applies your classification model to calculate the variables (“Nut or shell”).

On the right side you see a panel with the settings for this descriptor.

Go to the Table view

Under the Table press the Apply changes button.

The Analyse Tree is applied to the data in this project and the table will be updated with the results.

The columns with the blue header is showing the classification results from the model.

You can click in the table to see the class for each sample in the image.

The “Nut or shell” column with the colored square is showing the class for the sample based on its average spectrum.

The “Nut or shell” column with the small thumbnail image of the sample is showing the classification based on the spectrum for each pixel.

Collapse the left structure to view the results better. You can also drag the vertical line between the image and the table to adjust the image size.

Click the button above the preview image to add a legend with classes.

Expand the menu on the left, and select the Test group to see how it classified the objects in the test image.

Before we can do the real-time prediction we need to connect to the Prediktera simulated camera in settings.

Settings is always available from everywhere in the system.

Press “Settings” in the lower left corner. In this view you can set up a camera or a sample mover.

The “Prediktera Simulator camera” should be selected from the drop down list.

When it's connected it should look like this

Close Settings

With your workflow of Nuts Classification selected, Press “Recorder”

In the window that appears select “New Group”,

Name the group like “Realtime” or use default

Press “Add”.

We are now in the “Recorder” mode. Here you can do data acquisition from a camera and run real time analysis using your Workflow.

In the panel of the left side you can configure how to run your workflow. (For example, you can choose to uncheck “Include spectral raw data” to save space on your hard drive.)

Under Measurement Segmentation select the option Parallel

Click the red Play button on the left side to start the real time prediction.

As you can see the image is analyzed in real-time and the results are displayed in the table.

Click the Stop button

Press the green Done button to leave the Recorder mode and go back to the Workflow view.

Here you can see the results saved in the table.