Skip to main content
Skip table of contents

Intro to Breeze: Classification of nuts step 1

Welcome to this tutorial on Breeze! In this session, you'll become familiar with Breeze's user interface and workflow. We'll work with an example dataset to create a classification model that distinguishes between different classes in a hyperspectral image.

You will analyze hyperspectral images with samples of nuts (almond, hazelnut, pecan, and walnut) and shells for each nut type. The tutorial images contain samples of known nut types that will be used as a training data set and an image with a mix of samples that will be used as a test dataset.

Example on Hyperspectral image with original data and classified data

Hyperspectral image and classification of objects.

The steps in the tutorial are:

  1. Download tutorial image data

  2. Label your training data for classification of models

  3. Create a sample model to remove background pixels

  4. Create a PLS-DA classification model

  5. Apply your model to classify samples

  6. Real-time prediction

    1. Add a Workflow for real time analysis

    2. Start real time prediction

Download tutorial image data

Installation of Breeze: (skip this step if you already have installed Breeze).

After downloading Breeze, start the installation file (Prediktera_Breeze_Windows_2024.2.0.exe) with the shortcut found in your browser or in your download folder.

Image on shortcut link in browser

Shortcut link in browser

Image on shortcut link in download folder

Shortcut link in download folder

A Breeze installer wizard will appear. Follow the steps in the wizard to complete the installation.

Press Next

image-20240528-143458.png

You will be asked to select a camera supplier.

Select the camera you plan to use with Breeze for data aquisition.

For this tutorial select No external camera.

Press Next

image-20240528-144548.png

After starting Breeze for the first time a pop up will appear with links to our Help site with tutorials.

Press OK to close it.

Image on welcome and help pop up

Pop up with links to tutorials on our Help site

When the help pop up is not visible, you can find the tutorial on the Breeze Home view.

Click on Tutorial to start.

Image on tutorial link to click

Breeze Home view

Select “Nuts Classification” in the “Name” drop-down menu.


Press OK to start downloading the image data.

Image on prepared project to choose

Always choose a project to work on. That is done here i the home view of Breeze.

There are three different way to open the project…

  1. Double click on project

  2. Click once and choose Analyzer

  3. Right click and choose Analyzer

Image on how to open a project

The image data in this Project is organized into two Groups (folders) called “Train” and “Test” and includes eight training images (with either nuts or shells) and one test image.

You can click on a table row to see the preview image (pseudo-RGB) for each image.

Double click on the “Train” group to open it.

Image on project structure in Breeze

In the menu on the left side, you see all the individual images (called Measurements), in this group.

Click on the Pixel Explore tab to look closer at an individual image and its spectral data. In this example we have selected the first image “Almond”

Image on where to find pixel explorer

Collapse the left menu using the small arrow to see expand the “Pixel Explore” view.

By hovering the mouse over the image you can see the spectrum for the pixel at the position of the curser.

Hold down left mouse button and make a selection of an area of the image. The average spectrum for this area will be shown in the spectrum plot.

You can compare different areas by making additional selections.

In the meny on the right side are options for different Selection and Zoom tools. You can also zoom by using the scroll function on your mouse.

The spectral plot has 3 vertical lines showing the spectral bands used to generate the pseudo RGB image. Can can modify this by sliding the lines using your mouse.

Now let’s do a quick analysis of the spectral variation in the image, by clicking on the + in Model (PCA)

image-20240528-152722.png

Collapse left menu

image-20240528-154026.png

Menu on right side

Screenshot 2024-05-20 134500.png

Do a Model (PCA) analysis

A PCA model has been created based on all pixels in the image. Each point in the “Variance scatter” plot corresponds to a pixel in the image. The points in the scatter plot are clustered based on spectral similarity. The color of the points in the scatter plot is based on density (i.e. red = many points close to each other).

Visualization of PCA modelling

The “Max variance image” is colored by the variation in the 1st component of the PCA model (the X-axis in the scatter plot, t1), and visualizes the biggest spectral variation in the image. In this case, this is the difference between the sample (blue) and the background (orange).

Do a selection of a cluster of points in the scatter plot. These pixels are automatically highlighted in the image.

Screenshot 2024-05-20 141145.png

Go back to the project level by clicking on the Back button in the upper left corner.

back.png


Label your training data for classification of models

Select “Nuts-Classification” in the menu on the right side panel to select the project. This will enable you to see all images from both the “Train” and “Test” groups in Table.

We will now add a new class variable to your training data set so we can define the true values. This is done by creating a new column in the table.

Click “Add variable or Id” available at the bottom of the page.

Untitled.png

Select type “Category (Classification variable)” and write the name “Nut or shell” and press Add.

Screenshot 2024-05-20 145807.png

A column for “Nut or shell” has now been added to the Table

image-20240520-130543.png

TIP If you need to delete a column like Variables or IDs, right-click on the header for the column you want to delete and then press the “Delete” option that will appear.

Categorize all nuts, by pressing the Ctrl-key and select with the mouse cursor “Almonds”, “Hazelnut”, “Pecans” and “Walnuts” samples.

Right-click on one of the selected rows in the “Nut or Shell” column and write the class name “Nut” in the “New Class” field and pressSlect Add.

image-20240520-132535.png

Do the same thing again for the shell samples and categorize them with the new class “Shell”.

image-20240520-132728.png

Your table should look like this now.

You do not need to set a class to the Test image.

image-20240520-141831.png


Create a sample model to remove background pixels

You will now create a sample model that will be used to remove the background pixels and to automatically segment out the objects (nut and shell samples) in the images.

This sample model will be applied to all images in the project and will make it easy for us to train a classification model in next step.

Press “Add Sample model” button at the bottom of the screen:

add sample model.png

Choose a name for the sample model (or use the default name)

press “OK”

naming.jpg

A model wizard will appear.

In the first step - select the Measurements (images), that you will use in the model.

By default, all measurements are included, which is ok.


Press “Next”

stepbystep.png

Step two - Select Wavelengths (spectral bands) to use in the model.

On the left side you see a table of all spectral bands. On the right you have a spectral plot showing the average spectrum of each image. By default, all wavelengths are included as shown by being highlighted in green.

You can also see the Pre-treatment steps that can be applied to the data. By default “SNV (Standard Normal Variate correction)“ is selected (see Pretreatments | SNV-(Standard-Normal-Variate-correction) for more information).

Press “Next”

awavelength.png

Step 3 - Model, select the pixels to use to train the sample model.

A mosaic has been created of all images and a PCA model has been created from all the pixels in this mosaic.

image-20240527-080606.png

Select a region containing only nut or shell pixels by holding down the left mouse button and mark the area inside the object.

To make this easier you can use the mouse scroll function to zoom in.

The corresponding pixels are then selected in the scatter plot to the left.

Now you know that the nut and shell pixels are in the cluster on the right side in the Scatter plot.

In the scatter plot, select all pixels in the cluster on the right side (use the pixel density coloring red, yellow, green, and light blue as help).

The corresponding pixels are then selected in the image on the right side and should correspond to all nut and shell pixels.

Press the “Include Only” button in the menu:

ainclude.png

The plots are now updated and will contain mostly the nuts pixels.

To clean up the nuts pixels even more you can remove the pixels bordering each sample object.
Press the “Select - Border pixels” button.

In pop up - Use the default of 1 border pixel and press “OK”.

aborder.png

The border pixels have now been selected.

Exclude the border pixels by pressing the “Exclude” button

Press Next

aexclude.png

Step 4 - Critical Distance, set the Critical Distance threshold.

This is the distance to the sample model and will be used to determine if pixels are sample or not. The histogram is showing the distance to the model for all pixels in the images. Pixels on the left side of the red vertical line (critical distance) are inside the threshold.

acriticaldistance.png

Drag the red line to the right to move the threshold. As you can see from image to the right, more pixels are included when doing this.

The aim is to find a level where all nut pixels are included but not pixels from the background.

As a general recommendation, you can drag the red line to the bottom of the “valley” between the sample and the background bars in the histogram as shown below.

Press “Next”

Step 5 - Approve, the “Minimum area size” is used to automatically exclude smaller unwanted objects (for example dust or dirt).

Breeze calculates a suggested minimum area size for your data. In this example, any objects under 300 pixels will be excluded from the image. Depending on how you did the pixel selection in the previous step this value might vary. A value around 300 should be OK.

Press “Finish” to create the sample model and apply this to all images in the project.

image-20240527-090507.png

In the “Table” for the project Nuts_Classification, you can now see all the unique sample objects in the images after the sample model has been applied and the background pixels removed.

Collapse the left structure to view the results better.

anuttyclassification.png

Click on a sample in the “Nut or shell” column in the table to color all objects in the preview image based on the class.

image-20240527-094305.png

You can also click on the objects in the preview image to see where they are in the table.

image-20240527-093922.png

Press the “Explore” tab. A PCA model has been created based on the average spectrum for each sample. Each point in the scatter plot corresponds to a sample and the points are clustered based on spectral similarity. Select one or several points to see their average spectrum.

Screenshot 2024-05-27 114507.png

In the menu on the right click on the category name “Nut or shell” to color the scatter plot and preview the image based on the different classes (the red dots are the “Mix” samples where we had not entered any class).

(you can close the preview image with the arrow to make more space for the plots and graphs)..

anutorshell.png

Create a PLS-DA classification model

You will now use the average spectrum for each sample and the class type that you have set to train a classification model.

To create another model, go to the Model view by pressing the button on the left side of the screen.

amodel.png

In the menu on the left, you can see the Sample model that you created before.

To make an additional model Press the “Add” button in lower left corner.

On the window that appears, press the “Classification” tab, and then select “PLS-DA”.

Write a name for the model (or use the default name)

press “OK”.

aclassification.png

Step 1 - Variables

The Segmentation using the Sample model will be used.

The “Nut or shell” variable has been selected and will be used to build the model.

press “Next”.

image-20240527-110550.png

Step 2 - Samples

Select the samples that you want to include in the model.

By default, the measurements from the “Train” group have been included since they have entered class information.

The measurement “Mix” in the “Test” group has been excluded since there was no class information entered. This is OK.

Press “Next”

image-20240527-111320.png

Step 3 - Wavelength

By default, all wavelength bands are included. The graph on the right is showing the average spectrum for each sample.

By default “SNV (Standard Normal Variate correction)“ is used. (see Pretreatments | SNV-(Standard-Normal-Variate-correction) for more information).

Press next

image-20240529-133546.png

Step 4 - Model

A PLS-DA classification model has now been calculated.

FYI - The results on your screen might vary slightly from these result seen here due to how you set the Critical Distance threshold in the previous segmentation.

image-20240527-112909.png

The “Overview (Total for all Y)” graph is showing how good the PLS-DA model is.

It also shows the number of components used for the model. In this case, the autofit used six components. The R2 (model fit) and Q2 (prediction from cross-validation) using six components are around 0.98/0.97 indicating a very good model.

An R2 and Q2 value of 1.0 indicates a model explaining all the variation. A value of 0 indicates that no variation can be explained. In this example it looks like the R2 and Q2 do not increase much after the 3rd component so we could choose to only use 3 component.

But for the sake of simplicity will use the autofit model of 6 (or 5 components).

image-20240527-113042.png

The “Distance to model in X” and “Y” graphs show the distance to the model for each sample.

A high bar indicates that the sample might be an outlier (for the X distance the horizontal black line can be used as a guide.

image-20240527-113642.png

The “Score variance” scatter plot and the “Distance to model” graphs can be used to identify and exclude outliers.

In this example we are not going to remove any outliers. Press “Next”.

image-20240527-113752.png

Step 5 - Evaluate how good the model is.

The “Nut or shell” vs. “Ycalc.

Nut or shell” is showing how well the model can separate the two classes.

The “Variable overview” is showing the R2 and Q2 for each class.

Everything looks OK.

Press “Finish” to complete the model.

image-20240527-115829.png

The Classification PLS-DA model has now been saved as you can see in the menu on the left.

With this model selected press the “Classification” tab to see how many samples were correctly classified.

aclassification2.png

Each row is showing the correct class for the samples, and the columns are showing the classes that these samples are classified into by the model.

In this example, all the samples are correctly classified.

Screenshot 2024-05-27 140511.png

Close Model

aclose.png

Apply your model to classify samples

In this step, you will use the PLS-DA classification model (Classification - Nuts_classification, from the previous step) to analyze the images and classify the samples.

Press the Analyse Tree tab at the top of the screen.

Here you can make your own data processing workflow to automate the analysis of the images in your project.

The sample model we developed before is already in the Analysis Tree as the first processing step of your data (Measurements).

Press the “+“ symbol after “Object”

image-20240529-135700.png

A pop-up will appear where you can select the data processing function you want to apply.

Press Descriptor and then select Classification of Categories

image-20240529-140523.png

The “Nut or shell” PLS-DA model has now been added to the Analyse Tree.

When we apply this to our data, first, the “Measurement” (image) is analyzed by your sample model (“Sample - Nuts_Classification”) to find the sample “Object”.

For this object, it then applies your classification model to calculate the variables (“Nut or shell”).

On the right side you see a panel with the settings for this descriptor.

Go to the Table view

image-20240529-141000.png

Under the Table press the Apply changes button.

The Analyse Tree is applied to the data in this project and the table will be updated with the results.

The columns with the blue header is showing the classification results from the model.

You can click in the table to see the class for each sample in the image.

image-20240529-141818.png

Apply changes

image-20240529-142147.png

Click in table to see the class in image.

The “Nut or shell” column with the colored square is showing the class for the sample based on its average spectrum.

The “Nut or shell” column with the small thumbnail image of the sample is showing the classification based on the spectrum for each pixel.

Collapse the left structure to view the results better. You can also drag the vertical line between the image and the table to adjust the image size.

Click the button above the preview image to add a legend with classes.

image-20240529-142450.png

Expand the menu on the left, and select the Test group to see how it classified the objects in the test image.

image-20240529-143128.png

Real-time prediction

In addition to analyzing images that are already recorded on your hard drive, you can also use Breeze Recorder to analyze images in real-time directly from the camera.

If your computer is not connected to a camera, you can simulate this by using the “Camera simulator” in Breeze. With this, it will read images from your hard drive and analyze them continuously.

Add a Workflow for real time analysis

Press the Workflow button on the left side of the screen

In the Workflow view press Add button in the lower left corner and then select New. Give it name or use the default name.

Press OK

In the Analyse Tree tab you can see that it is using as a default the Analyse Tree we just created in the previous step.

image-20240529-144921.png

Press Workflow button

image-20240529-145304.png

Press Add button plus OK button

Before we can do the real-time prediction we need to connect to the Prediktera simulated camera in settings.

Settings is always available from everywhere in the system.

PressSettings” in the lower left corner. In this view you can set up a camera or a sample mover.

The “Prediktera Simulator camera” should be selected from the drop down list.

image-20240527-135457.png

When it's connected it should look like this

Close Settings

image-20240527-135359.png

Start real time prediction

With your workflow of Nuts Classification selected, Press “Recorder”

arecorder.png

In the window that appears select “New Group”,

Name the group like “Realtime” or use default

Press “Add”.

image-20240527-140645.png

We are now in the “Recorder” mode. Here you can do data acquisition from a camera and run real time analysis using your Workflow.

In the panel of the left side you can configure how to run your workflow. (For example, you can choose to uncheck “Include spectral raw data” to save space on your hard drive.)

Under Measurement Segmentation select the option Parallel

Click the red Play button on the left side to start the real time prediction.

image-20240529-151846.png

As you can see the image is analyzed in real-time and the results are displayed in the table.

Click the Stop button

image-20240527-141843.png

Press the green Done button to leave the Recorder mode and go back to the Workflow view.

Here you can see the results saved in the table.

image-20240527-141943.png

Done button

image-20240527-143145.png

Workflow view with saved result

Nice job! You have reached the end of step 1 of the “Classification of Nuts” tutorial.
If you would like to learn more about classification analysis please try tutorial step 2 to learn some additional features.


Nut classification step 2

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.