Initial commit

README.md

@@ -1,12 +1,62 @@
----
-title: Afm Analysis Web
-emoji: 🌖
-colorFrom: gray
-colorTo: indigo
-sdk: gradio
-sdk_version: 4.31.3
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# **Stratum corneum nanotexture feature detection using deep learning and spatial analysis: a non-invasive tool for skin barrier assessment**
+
+<img src="./source/Overview.png" alt="Data Processing" width="95%" />
+
+This repository presents an automated approach for the data processing of Atomic Force Microscopy (AFM), enabling the construction of an extensive database for further academic investigation and visualization. The program seamlessly integrates critical steps, including the conversion of raw AFM data into PNG files, the utilization of computer vision techniques, and the implementation of state-of-the-art deep learning algorithms for accurate detection of circular nano objects (CNOs) and classification of various skin diseases. In addition, the algorithm incorporates the grid search method to determine the optimal hyperparameter settings, ensuring optimal performance and enhancing the reliability of the results.
+
+## **Dependencies**
+- Python 3.9+
+- matplotlib
+- numpy
+- opencv-python
+- scipy
+- scikit-image
+- ultralytics
+- customtkinter
+- scikit-learn
+- customtkinter
+
+## **Directories**
+- `AD_Assessment_GUI.zip` contains a cross-platform executable GUI, sample data, and a tutorial video.
+- Folder `corneocyte dataset` contains original corneocyte nanotexture images and annotated images for training AI models.
+- Folder `models` contains our fine-tuned YOLOv8-{N,S,M,L,X} and YOLOv9-{C,E} models.
+
+## **Usage**
+1. Execution via cross-platform executable GUI
+    - Unzip `AD_Assessment_GUI.zip`
+    - Run `AD_Assessment_GUI.exe`
+    - Analysis results will be saved within the selected path in a folder titled `CNO_Detection`
+
+2. Execution via python script
+    - Install packages in terminal:
+        ```    
+        pip install -r requirements.txt
+        ```
+    - Run `AD_Assessment_GUI.py`
+    - Analysis results will be saved within the selected path in a folder titled `CNO_Detection`
+
+## **Executable**
+
+1. Install PyInstaller in terminal:
+                
+    ```    
+    pip install pyinstaller
+    ```
+   
+2. Run command in terminal:
+
+    ```    
+    pyinstaller --onedir .\AD_Assessment_GUI.py
+    ```
+
+## **Contributions**
+
+[1] Liao, H-S., Wang, J-H., Raun, E., Nørgaard, L. O., Dons, F. E., & Hwu, E. E-T. (2022). Atopic Dermatitis Severity Assessment using High-Speed Dermal Atomic Force Microscope. Abstract from AFM BioMed Conference 2022, Nagoya-Okazaki, Japan.
+
+[2] Pereda, J., Liao, H-S., Werner, C., Wang, J-H., Huang, K-Y., Raun, E., Nørgaard, L. O., Dons, F. E., & Hwu, E. E. T. (2022). Hacking Consumer Electronics for Biomedical Imaging. Abstract from 5th Global Conference on Biomedical Engineering & Annual Meeting of TSBME, Taipei, Taiwan, Province of China.
+
+[3] Liao, H. S., Akhtar, I., Werner, C., Slipets, R., Pereda, J., Wang, J. H., Raun, E., Nørgaard, L. O., Dons, F. E., & Hwu, E. E. T. (2022). Open-source controller for low-cost and high-speed atomic force microscopy imaging of skin corneocyte nanotextures. HardwareX, 12, [e00341]. https://doi.org/10.1016/j.ohx.2022.e00341
+
+----
+
+### Contact: [Jen-Hung Wang](mailto:[email protected]) / [Professor En-Te Hwu](mailto:[email protected])

models/YOLOv8-L_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94088cd0fcba7758323f2b26aca84cf8a5b368917c39282f9a5da873baefa3d0
+size 87619390

models/YOLOv8-M_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:177d75ec783ccd7159d6c1c7d3eb3e066e37123afb7882b57f91042548a146b9
+size 52000800

models/YOLOv8-N_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e9cfcf31a8160ee9eb7ec8f5b54531224af571b516c80bd567858a3af26f1a2
+size 6260697

models/YOLOv8-S_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93627f63d6076b2ff6967495ebfb9f812bb52a0f071891d30e34952f4489a9e0
+size 22529177

models/YOLOv8-X_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff2b1c64c355730334537d99b59576ff8596126ce6fc8d8f0274d48b49c0b7f4
+size 136690238

models/YOLOv9-C_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2410809a30ec6a8acac2da28b4780aac25a1d97332648d815607c1427345737e
+size 102767066

models/YOLOv9-E_CNO_Detection.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5da00cae99ae6ea6676556e8ea11ce06002a0aa01d4b59eebb54b24e2fa0e289
+size 139940794

requirements.txt

@@ -0,0 +1,10 @@
+gradio
+matplotlib
+numpy
+pandas
+Pillow
+scikit_learn
+scipy
+skimage
+ultralytics
+opencv-python

utils/CNO_KDE_Integration.py

@@ -0,0 +1,344 @@
+# Copyright 2024 Jen-Hung Wang, IDUN Section, Department of Health Technology, Technical University of Denmark (DTU)
+
+import time
+import sys
+import warnings
+import csv
+import cv2
+import math
+from pathlib import Path
+from utils.growcut import *
+from ultralytics import YOLO
+from sklearn.neighbors import KernelDensity
+from sklearn.model_selection import GridSearchCV
+
+warnings.filterwarnings('ignore')
+DIR_NAME = Path(os.path.dirname(__file__)).parent
+np.set_printoptions(threshold=sys.maxsize)
+# Use GPU
+# torch.cuda.set_device(0) # Set to your desired GPU number
+
+# Model Path
+DETECTION_MODEL_n = os.path.join(DIR_NAME, 'models', 'YOLOv8-N_CNO_Detection.pt')
+DETECTION_MODEL_s = os.path.join(DIR_NAME, 'models', 'YOLOv8-S_CNO_Detection.pt')
+DETECTION_MODEL_m = os.path.join(DIR_NAME, 'models', 'YOLOv8-M_CNO_Detection.pt')
+DETECTION_MODEL_l = os.path.join(DIR_NAME, 'models', 'YOLOv8-L_CNO_Detection.pt')
+DETECTION_MODEL_x = os.path.join(DIR_NAME, 'models', 'YOLOv8-X_CNO_Detection.pt')
+# DETECTION_MODEL_c = os.path.join(DIR_NAME, 'models', 'YOLOv9-C_CNO_Detection.pt')
+# DETECTION_MODEL_e = os.path.join(DIR_NAME, 'models', 'YOLOv9-E_CNO_Detection.pt')
+
+def numcat(arr):
+    arr_size = arr.shape[0]
+    arr_cat = np.empty([arr_size, 1], dtype=np.int32)
+    for i in range(arr.shape[0]):
+        arr_cat[i] = arr[i][0] * 1000 + arr[i][1]
+    return arr_cat
+
+
+def cno_detection(source, kde_dir, conf, cno_model, file_list, model_type):
+
+    # Declare Parameters
+    cno_col = []
+    total_layer_area = []
+    total_layer_cno = []
+    total_layer_density = []
+    avg_area_col = []
+    total_area_col = []
+
+    detection_results = cno_model.predict(source, save=False, save_txt=False, iou=0.5, conf=conf, max_det=1200)
+
+    # CNO Analysis
+    for idx, result in enumerate(detection_results):
+        CNO = len(result.boxes)
+        single_layer_area = []
+        single_layer_cno = []
+        single_layer_density = []
+        total_area = 0
+        if CNO < 5:
+            avg_area_col.append(np.nan)
+            total_area_col.append(np.nan)
+            nan_arr = np.empty([25])
+            nan_arr[:] = np.nan
+            total_layer_area.append(nan_arr)
+            total_layer_cno.append(nan_arr)
+            total_layer_density.append(nan_arr)
+        else:
+            CNO_coor = np.empty([CNO, 2], dtype=int)
+            for j in range(CNO):
+                w = result.boxes.xywh[j][2]
+                h = result.boxes.xywh[j][3]
+                area = (math.pi * w * h / 4) * 20 * 20 / (512 * 512)
+                total_area += area
+                bbox_img = result.orig_img
+                x = round(result.boxes.xywh[j][0].item())
+                y = round(result.boxes.xywh[j][1].item())
+
+                x1 = round(result.boxes.xyxy[j][0].item())
+                y1 = round(result.boxes.xyxy[j][1].item())
+                x2 = round(result.boxes.xyxy[j][2].item())
+                y2 = round(result.boxes.xyxy[j][3].item())
+
+                CNO_coor[j] = [x, y]
+                bbox_img = cv2.rectangle(bbox_img,
+                                         (x1, y1),
+                                         (x2, y2),
+                                         (0, 255, 0), 1)
+
+            avg_area = total_area / CNO
+            avg_area_col.append(round(avg_area.item(), 4))
+            total_area_col.append(round(total_area.item(), 4))
+
+            cv2.imwrite(os.path.join(kde_dir, '{}_{}_{}_bbox.png'.format(file_list[idx], model_type, conf)),
+                        bbox_img)
+
+            kde = KernelDensity(metric='euclidean', kernel='gaussian', algorithm='ball_tree')
+
+            # Finding Optimal Bandwidth
+            ti = time.time()
+            if CNO < 7:
+                fold = CNO
+            else:
+                fold = 7
+            gs = GridSearchCV(kde, {'bandwidth': np.linspace(20, 60, 41)}, cv=fold)
+            cv = gs.fit(CNO_coor)
+            bw = cv.best_params_['bandwidth']
+            tf = time.time()
+            print("Finding optimal bandwidth={:.2f} ({:n}-fold cross-validation): {:.2f} secs".format(bw, cv.cv,
+                                                                                                      (tf - ti)))
+            kde.bandwidth = bw
+            _ = kde.fit(CNO_coor)
+
+            xgrid = np.arange(0, bbox_img.shape[1], 1)
+            ygrid = np.arange(0, bbox_img.shape[0], 1)
+            xv, yv = np.meshgrid(xgrid, ygrid)
+            xys = np.vstack([xv.ravel(), yv.ravel()]).T
+            gdim = xv.shape
+            zi = np.arange(xys.shape[0])
+            zXY = xys
+            z = np.exp(kde.score_samples(zXY))
+            zg = -9999 + np.zeros(xys.shape[0])
+            zg[zi] = z
+
+            xyz = np.hstack((xys[:, :2], zg[:, None]))
+            x = xyz[:, 0].reshape(gdim)
+            y = xyz[:, 1].reshape(gdim)
+            z = xyz[:, 2].reshape(gdim)
+            levels = np.linspace(0, z.max(), 26)
+            print("levels", levels)
+
+            for j in range(len(levels) - 1):
+                area = np.argwhere(z >= levels[j])
+                area_concatenate = numcat(area)
+                CNO_concatenate = numcat(CNO_coor)
+                ecno = np.count_nonzero(np.isin(area_concatenate, CNO_concatenate))
+                layer_area = area.shape[0]
+                if layer_area == 0:
+                    density = np.round(0.0, 4)
+                else:
+                    density = np.round((ecno / layer_area) * 512 * 512 / 400, 4)
+                print("Level {}: Area={}, CNO={}, density={}".format(j, layer_area, ecno, density))
+                single_layer_area.append(layer_area)
+                single_layer_cno.append(ecno)
+                single_layer_density.append(density)
+
+            total_layer_area.append(single_layer_area)
+            total_layer_cno.append(single_layer_cno)
+            total_layer_density.append(single_layer_density)
+
+            # Plot CNO Distribution
+            plt.contourf(x, y, z, levels=levels, cmap=plt.cm.bone)
+            plt.axis('off')
+            # plt.gcf().set_size_inches(8, 8)
+            plt.gcf().set_size_inches(8 * (gdim[1] / gdim[0]), 8)
+            plt.gca().invert_yaxis()
+            plt.xlim(0, gdim[1] - 1)
+            plt.ylim(gdim[0] - 1, 0)
+            plt.savefig(os.path.join(kde_dir, '{}_{}_{}_KDE.png'.format(file_list[idx], model_type, conf)),
+                        bbox_inches='tight', pad_inches=0)
+            plt.clf()
+
+            plt.scatter(CNO_coor[:, 0], CNO_coor[:, 1], s=10)
+            plt.xlim(0, gdim[1] - 1)
+            plt.ylim(0, gdim[0] - 1)
+            plt.axis('off')
+            plt.gcf().set_size_inches(8, 8)
+            plt.gcf().set_size_inches(8 * (gdim[1] / gdim[0]), 8)
+            plt.gca().invert_yaxis()
+            plt.savefig(os.path.join(kde_dir, '{}_{}_{}_Spatial.png'.format(file_list[idx], model_type, conf)),
+                        bbox_inches='tight', pad_inches=0)
+            plt.clf()
+        cno_col.append(CNO)
+
+    return cno_col, avg_area_col, total_area_col, total_layer_area, total_layer_cno, total_layer_density
+
+
+def cno_detect(folder_dir, model, conf):
+
+    if model == 'YOLOv8-N':
+        CNO_model = YOLO(DETECTION_MODEL_n)
+    elif model == 'YOLOv8-S':
+        CNO_model = YOLO(DETECTION_MODEL_s)
+    elif model == 'YOLOv8-M':
+        CNO_model = YOLO(DETECTION_MODEL_m)
+    elif model == 'YOLOv8-L':
+        CNO_model = YOLO(DETECTION_MODEL_l)
+    else:
+        CNO_model = YOLO(DETECTION_MODEL_x)
+    """  
+    elif model == 'YOLOv9-C':
+        CNO_model = YOLO(DETECTION_MODEL_c)
+    else:
+        CNO_model = YOLO(DETECTION_MODEL_e)
+    """
+
+    # Search folder path
+    folder = folder_dir.split(os.sep)[-1]
+
+    print("Analyzing Folder", folder)
+
+    # Extract folder information
+    folder_info = folder.split('_')
+    if folder_info[2][0:2] == "TL":
+        Country = folder_info[0]
+        AD_severity = folder_info[1]
+        TLSS = int(folder_info[2].strip("TL"))
+        if TLSS == 0:
+            lesional = False
+        else:
+            lesional = True
+        Number = int(folder_info[-1].strip("No."))
+        AD_group = AD_severity.strip("G")
+    else:
+        Country = None
+        TLSS = None
+        lesional = None
+        Number = None
+        AD_group = None
+
+    run_growcut = True
+    timestr = time.strftime("%Y%m%d-%H%M%S")
+
+    CNO_list = []
+    Area_sum = []
+    Area_avg = []
+
+    file_list = []
+    growcut_list = []
+
+    growcut_path = os.path.join(folder_dir, "CNO_Detection", "GrowCut")
+    original_png_path = os.path.join(folder_dir, "CNO_Detection", "Image", "Original")
+    enhanced_png_path = os.path.join(folder_dir, "CNO_Detection", "Image", "Enhanced")
+    kde_png_path = os.path.join(folder_dir, "CNO_Detection", "Image", "KDE")
+    save_dir = os.path.join(folder_dir, "CNO_Detection", "Result")
+    print("Save Path:", save_dir)
+
+    try:
+        os.makedirs(growcut_path, exist_ok=True)
+        os.makedirs(original_png_path, exist_ok=True)
+        os.makedirs(enhanced_png_path, exist_ok=True)
+        os.makedirs(kde_png_path, exist_ok=True)
+        if not os.listdir(enhanced_png_path):
+            print("Directory is empty")
+            run_growcut = True
+        else:
+            print("Directory is not empty")
+            run_growcut = False
+        os.makedirs(save_dir, exist_ok=True)
+    except OSError as error:
+        print("Directory can not be created")
+
+    encyc = []
+    walk = os.walk(folder_dir)
+    for d, sd, files in walk:
+        directory = d.split(os.sep)[-1]
+        for fn in files:
+            if fn[0:2] != "._" and fn[-10:].lower() == '_trace.bcr' and directory == folder:
+                encyc.append(d + os.sep + fn)
+    encyc.sort()
+
+    # GrowCut Detection
+    if run_growcut:
+        for i, fn in enumerate(encyc):
+            file, gc_CNO = treat_one_image(fn, growcut_path, original_png_path, enhanced_png_path)
+            file_list.append(file)
+            growcut_list.append(gc_CNO)
+            print(i, end=' ')
+    else:
+        for i, fn in enumerate(encyc):
+            file_list.append(os.path.split(fn)[1][0:-10])
+
+    # CNO Detection & AD Classification
+    print("Model", model)
+    print("Conf", conf)
+
+    # Make Function
+    cno_col, avg_area_col, total_area_col, layer_area, layer_cno, layer_density = cno_detection(enhanced_png_path,
+                                                                                                kde_png_path,
+                                                                                                conf, CNO_model,
+                                                                                                file_list, model)
+    CNO_list.append(cno_col)
+    Area_sum.append(total_area_col)
+    Area_avg.append(avg_area_col)
+
+    Layer_area = layer_area
+    Layer_cno = layer_cno
+    Layer_density = layer_density
+
+    # Write CSV
+    # open the file in the write mode
+    f = open(save_dir + os.sep + '{}_{}_{}_{}_.csv'.format(folder, timestr, model, conf), 'w')
+    header = ['File', 'Country', 'Group', 'No.', 'TLSS', 'Lesional',
+
+              'Layer_Area_0', 'Layer_Area_1', 'Layer_Area_2', 'Layer_Area_3', 'Layer_Area_4',
+              'Layer_Area_5', 'Layer_Area_6', 'Layer_Area_7', 'Layer_Area_8', 'Layer_Area_9',
+              'Layer_Area_10', 'Layer_Area_11', 'Layer_Area_12', 'Layer_Area_13', 'Layer_Area_14',
+              'Layer_Area_15', 'Layer_Area_16', 'Layer_Area_17', 'Layer_Area_18', 'Layer_Area_19',
+              'Layer_Area_20', 'Layer_Area_21', 'Layer_Area_22', 'Layer_Area_23', 'Layer_Area_24',
+
+              'Layer_CNO_0', 'Layer_CNO_1', 'Layer_CNO_2', 'Layer_CNO_3', 'Layer_CNO_4',
+              'Layer_CNO_5', 'Layer_CNO_6', 'Layer_CNO_7', 'Layer_CNO_8', 'Layer_CNO_9',
+              'Layer_CNO_10', 'Layer_CNO_11', 'Layer_CNO_12', 'Layer_CNO_13', 'Layer_CNO_14',
+              'Layer_CNO_15', 'Layer_CNO_16', 'Layer_CNO_17', 'Layer_CNO_18', 'Layer_CNO_19',
+              'Layer_CNO_20', 'Layer_CNO_21', 'Layer_CNO_22', 'Layer_CNO_23', 'Layer_CNO_24',
+
+              'Layer_Density_0', 'Layer_Density_1', 'Layer_Density_2', 'Layer_Density_3',
+              'Layer_Density_4', 'Layer_Density_5', 'Layer_Density_6', 'Layer_Density_7',
+              'Layer_Density_8', 'Layer_Density_9', 'Layer_Density_10', 'Layer_Density_11',
+              'Layer_Density_12', 'Layer_Density_13', 'Layer_Density_14', 'Layer_Density_15',
+              'Layer_Density_16', 'Layer_Density_17', 'Layer_Density_18', 'Layer_Density_19',
+              'Layer_Density_20', 'Layer_Density_21', 'Layer_Density_22', 'Layer_Density_23',
+              'Layer_Density_24',
+
+              'AVG_Area', 'AVG_Size']
+
+    writer = csv.writer(f)
+    writer.writerow(header)
+
+    for i in range(len(file_list)):
+        data = [file_list[i], Country, AD_group, Number, TLSS, lesional,
+
+                Layer_area[i][0], Layer_area[i][1], Layer_area[i][2], Layer_area[i][3], Layer_area[i][4],
+                Layer_area[i][5], Layer_area[i][6], Layer_area[i][7], Layer_area[i][8], Layer_area[i][9],
+                Layer_area[i][10], Layer_area[i][11], Layer_area[i][12], Layer_area[i][13],
+                Layer_area[i][14], Layer_area[i][15], Layer_area[i][16], Layer_area[i][17],
+                Layer_area[i][18], Layer_area[i][19], Layer_area[i][20], Layer_area[i][21],
+                Layer_area[i][22], Layer_area[i][23], Layer_area[i][24],
+
+                Layer_cno[i][0], Layer_cno[i][1], Layer_cno[i][2], Layer_cno[i][3], Layer_cno[i][4],
+                Layer_cno[i][5], Layer_cno[i][6], Layer_cno[i][7], Layer_cno[i][8], Layer_cno[i][9],
+                Layer_cno[i][10], Layer_cno[i][11], Layer_cno[i][12], Layer_cno[i][13], Layer_cno[i][14],
+                Layer_cno[i][15], Layer_cno[i][16], Layer_cno[i][17], Layer_cno[i][18], Layer_cno[i][19],
+                Layer_cno[i][20], Layer_cno[i][21], Layer_cno[i][22], Layer_cno[i][23], Layer_cno[i][24],
+
+                Layer_density[i][0], Layer_density[i][1], Layer_density[i][2], Layer_density[i][3],
+                Layer_density[i][4], Layer_density[i][5], Layer_density[i][6], Layer_density[i][7],
+                Layer_density[i][8], Layer_density[i][9], Layer_density[i][10], Layer_density[i][11],
+                Layer_density[i][12], Layer_density[i][13], Layer_density[i][14], Layer_density[i][15],
+                Layer_density[i][16], Layer_density[i][17], Layer_density[i][18], Layer_density[i][19],
+                Layer_density[i][20], Layer_density[i][21], Layer_density[i][22], Layer_density[i][23],
+                Layer_density[i][24],
+
+                Area_sum[0][i], Area_avg[0][i]]
+        writer.writerow(data)
+    f.close()
+

utils/growcut.py

@@ -0,0 +1,206 @@
+import os
+import re
+import numpy as np
+import matplotlib.pyplot as plt
+from matplotlib.widgets import MultiCursor
+from scipy import ndimage
+from matplotlib import cm
+from PIL import Image, ImageDraw, ImageFont
+from skimage import morphology
+from skimage.measure import regionprops
+
+
+def comp(*ims, figsize=(20, 10)):
+    N = len(ims)
+    ncols = {1: 1, 2: 2, 3: 3, 4: 2, 5: 3, 6: 3, 7: 4, 8: 4, 9: 3}
+    nrows = {1: 1, 2: 1, 3: 1, 4: 2, 5: 2, 6: 2, 7: 2, 8: 2, 9: 3}
+    fig, axes = plt.subplots(ncols=ncols[N], nrows=nrows[N], sharex=True, sharey=True, figsize=figsize)
+    fig.subplots_adjust(wspace=0.01, hspace=0.01)
+    axes = axes.ravel()
+    cursor = MultiCursor(fig.canvas, axes,
+                         horizOn=True, vertOn=True, color='red', linewidth=1)
+    for i in range(N):
+        axes[i].imshow(ims[i])
+    return fig, axes, cursor
+
+
+def load_im(fn):
+    f = open(fn, 'rb')
+    a = f.read()
+    f.close()
+    aa = str(a[:2048])
+    xpix = int(re.findall('xpixels\s?=\s?([0-9]*)', aa)[0])
+    ypix = int(re.findall('ypixels\s?=\s?([0-9]*)', aa)[0])
+    a = a[2048:]
+
+    words = [a[k * 2:k * 2 + 2] for k in range(xpix * ypix)]
+    arr = [int.from_bytes(words[k], byteorder='little', signed=True) for k in range(len(words))]
+    im = np.array(arr).reshape((ypix, xpix))
+
+    im = (im.T - np.mean(im, axis=1) +
+          np.mean(ndimage.gaussian_filter(im, 10), axis=1)).T  # palliate horizontal artifact
+    im = im - np.min(im)
+    im = im / np.max(im)  # normalize to 0.0-1.0
+    return im
+
+
+def G(x):
+    return 1 - np.abs(x) ** .5
+
+
+def growcut(land, labels, strength, maxiter=5):
+    Ni, Nj = land.shape
+    sidei, sidej = np.arange(Ni), np.arange(Nj)
+    ij = np.dstack(np.meshgrid(sidei, sidej))[:, :, ::-1]
+    iijj = np.tile(ij, (9, 1, 1, 1))
+    for i, k in enumerate(((0, 0), (1, 0), (1, 1), (0, 1), (-1, 1), (-1, 0), (-1, -1), (0, -1), (1, -1))):
+        iijj[i, :, :, :] += np.array(k)
+        iijj[i, :, :, 0] = iijj[i, :, :, 0].clip(0, land.shape[0] - 1)
+        iijj[i, :, :, 1] = iijj[i, :, :, 1].clip(0, land.shape[1] - 1)
+    neigh_slice = np.s_[iijj[:, :, :, 0], iijj[:, :, :, 1]]
+
+    this_labels = labels * 1
+    this_strength = strength * 1
+
+    neigh_val = land[neigh_slice]
+    jump_diff = land - neigh_val
+    g = G(jump_diff)
+
+    for i in range(maxiter):
+        # print(np.sum(this_labels), end=' ')
+        neigh_lab = this_labels[neigh_slice] * 1
+        neigh_str = this_strength[neigh_slice] * 1
+
+        attack_force = g * neigh_str
+
+        new_layer = np.argmax(attack_force, axis=0)
+        new_lab = neigh_lab[new_layer, iijj[0, :, :, 0], iijj[0, :, :, 1]] * 1
+        new_strength = attack_force[new_layer, iijj[0, :, :, 0], iijj[0, :, :, 1]] * 1
+
+        this_labels = new_lab
+        this_strength = new_strength
+
+    return this_labels, this_strength
+
+
+def pyramid_contrast(im):
+    oom = []
+    ms = []
+    for d in (9, 15):  # (9, 11, 13, 15, 17,25):#(3, 6, 9, 12, 15, 18, 21):
+        disk = morphology.disk(d)
+        m = ndimage.percentile_filter(im, 10, footprint=disk)
+        M = ndimage.percentile_filter(im, 90, footprint=disk)
+        om = (im - m) / (M - m)
+        om = np.nan_to_num(om).clip(0, 1)
+        oom.append(om)
+        ms.append(M - m)
+    oom = np.array(oom)
+    # ms = np.array(ms)
+    land = np.mean(oom, axis=0)
+    return land
+
+
+def segmentate(land, alpha=0.7, beta=0.6):
+    if alpha < beta:
+        print("alpha must be greater than beta")
+        assert False
+    foreground = ndimage.binary_erosion(land > alpha, iterations=1)
+    background = land < beta
+
+    lab = ndimage.label(foreground)[0]
+    lab[lab > 0] += 1
+    lab[background] = 1
+
+    strength = (lab > 1) * 1. + (lab == 1) * 1.
+    this_labels, this_strength = growcut(land,
+                                         lab, strength, maxiter=25)
+    w = (this_labels != np.roll(this_labels, 1, axis=0)) + (this_labels != np.roll(this_labels, 1, axis=1))
+
+    b = w * 0
+    lab2 = ndimage.label(~w)[0]
+    for l in np.unique(lab2)[1:]:
+        if np.sum(foreground[l == lab2]) > 0:
+            b[l == lab2] = 1
+
+    lab2 = ndimage.label(ndimage.binary_dilation(this_labels > 1))[0]
+    return lab2
+
+
+def filter_objects(lab2, max_eccentricity=0.93, min_size=10, max_size=200, min_convex_coverage=0.8):
+    props = regionprops(lab2)  # object metrics
+    b = lab2 * 0.
+    for i in np.unique(lab2)[1:]:
+        ind = i - 1
+        e = props[ind].eccentricity
+        s = props[ind].area
+        c = s * 1 / props[ind].convex_area
+        # filter objects by eccentricity, size, ans convex hull coverage
+        if e < max_eccentricity and (min_size < s < max_size) and c > min_convex_coverage:
+            lev = 1
+        else:
+            lev = 2
+        b[lab2 == i] = lev
+    return b
+
+
+def present(im, land, b):
+    original_im = cm.afmhot(im)[:, :, :3]
+    # Contrast Level 0.5
+    resim = 0.5 * land + (1 - 0.5) * im
+    enhanced_im = cm.afmhot(resim)[:, :, :3]
+    monochrome_land = np.tile(land, (3, 1, 1)).transpose((1, 2, 0))
+    detected = b == 1
+    detected = ndimage.binary_dilation(detected)  # * (~ndimage.binary_erosion(detected))
+    filtered = b == 2
+    filtered = ndimage.binary_dilation(filtered)  # * (~ndimage.binary_erosion(filtered))
+
+    monochrome_land[detected] *= np.array([.3, 1, .3])
+    monochrome_land[filtered] *= np.array([1, .6, .6])
+
+    newim = np.hstack((enhanced_im, monochrome_land))
+    newim = np.dstack((newim, newim[:, :, 0] * 0 + 1))
+
+    base = Image.fromarray((newim * 255).astype(np.uint8))
+    original_im = Image.fromarray((original_im * 255).astype(np.uint8))
+    enhanced_im = Image.fromarray((enhanced_im * 255).astype(np.uint8))
+
+    # make a blank image for the text, initialized to transparent text color
+    txt = Image.new("RGBA", base.size, (255, 255, 255, 0))
+
+    # get a font
+    # fnt = ImageFont.truetype("/usr/share/fonts/truetype/freefont/FreeMonoBold.ttf", 30, encoding="unic")
+    fnt = ImageFont.load_default()
+    # get a drawing context
+    d = ImageDraw.Draw(txt)
+    ct = np.max(ndimage.label(b == 1)[0]) - 1
+    d.text((600, 10), "CNOs: {:d}".format(ct), font=fnt, fill=(255, 50, 50, 255))
+    out = Image.alpha_composite(base, txt)
+
+    return out, original_im, enhanced_im, ct
+
+
+def treat_one_image(fn, growcut_path, original_png_path, enhanced_png_path):
+
+    file_list = []
+    growcut_list = []
+
+    # load data
+    im = load_im(fn)
+
+    # pyramid contrast
+    land = pyramid_contrast(im)
+
+    # detect objects
+    lab2 = segmentate(land, alpha=.75, beta=0.7)
+
+    # visualize
+    b = filter_objects(lab2, max_eccentricity=0.967, min_size=30, max_size=200, min_convex_coverage=0.5)
+    growcut_im, original_im, enhanced_im, ct = present(im, land, b)
+
+    file_name = os.path.split(fn)[1][0:-10]
+
+    original_im.save(os.path.join(original_png_path, file_name) + '.png')
+    enhanced_im.save(os.path.join(enhanced_png_path, file_name) + '.png')
+    growcut_im.save(os.path.join(growcut_path, file_name) + '.png')
+
+    return file_name, ct

web_app.py

@@ -0,0 +1,210 @@
+import os
+import cv2
+import pandas as pd
+import PIL.Image as Image
+import gradio as gr
+import numpy as np
+import math
+from pathlib import Path
+from ultralytics import ASSETS, YOLO
+
+DIR_NAME = Path(os.path.dirname(__file__))
+DETECTION_MODEL_n = os.path.join(DIR_NAME, 'models', 'YOLOv8-N_CNO_Detection.pt')
+DETECTION_MODEL_s = os.path.join(DIR_NAME, 'models', 'YOLOv8-S_CNO_Detection.pt')
+DETECTION_MODEL_m = os.path.join(DIR_NAME, 'models', 'YOLOv8-M_CNO_Detection.pt')
+DETECTION_MODEL_l = os.path.join(DIR_NAME, 'models', 'YOLOv8-L_CNO_Detection.pt')
+DETECTION_MODEL_x = os.path.join(DIR_NAME, 'models', 'YOLOv8-X_CNO_Detection.pt')
+
+# MODEL = os.path.join(DIR_NAME, 'models', 'YOLOv8-M_CNO_Detection.pt')
+# model = YOLO(MODEL)
+# cno_df = pd.DataFrame()
+
+def predict_image(name, model, img, conf_threshold, iou_threshold):
+    """Predicts and plots labeled objects in an image using YOLOv8 model with adjustable confidence and IOU thresholds."""
+    gr.Info("Starting process")
+    # gr.Warning("Name is empty")
+    if name == "":
+        gr.Warning("Name is empty")
+
+    if model == 'YOLOv8-N':
+        CNO_model = YOLO(DETECTION_MODEL_n)
+    elif model == 'YOLOv8-S':
+        CNO_model = YOLO(DETECTION_MODEL_s)
+    elif model == 'YOLOv8-M':
+        CNO_model = YOLO(DETECTION_MODEL_m)
+    elif model == 'YOLOv8-L':
+        CNO_model = YOLO(DETECTION_MODEL_l)
+    else:
+        CNO_model = YOLO(DETECTION_MODEL_x)
+
+    results = CNO_model.predict(
+        source=img,
+        conf=conf_threshold,
+        iou=iou_threshold,
+        show_labels=False,
+        show_conf=False,
+        imgsz=512,
+        max_det=1200
+    )
+
+    cno_count = []
+    cno_image = []
+    file_name = []
+
+    # print("deb", img)
+
+    for idx, result in enumerate(results):
+        cno = len(result.boxes)
+        cno_coor = np.empty([cno, 2], dtype=int)
+        file_label = img[idx].split(os.sep)[-1]
+        for j in range(cno):
+            # w = r.boxes.xywh[j][2]
+            # h = r.boxes.xywh[j][3]
+            # area = (math.pi * w * h / 4) * 20 * 20 / (512 * 512)
+            # total_area += area
+            # bbox_img = r.orig_img
+            x = round(result.boxes.xywh[j][0].item())
+            y = round(result.boxes.xywh[j][1].item())
+
+            x1 = round(result.boxes.xyxy[j][0].item())
+            y1 = round(result.boxes.xyxy[j][1].item())
+            x2 = round(result.boxes.xyxy[j][2].item())
+            y2 = round(result.boxes.xyxy[j][3].item())
+
+            cno_coor[j] = [x, y]
+            cv2.rectangle(result.orig_img, (x1, y1), (x2, y2), (0, 255, 0), 1)
+        im_array = result.orig_img
+        cno_image.append([Image.fromarray(im_array[..., ::-1]), file_label])
+        cno_count.append(cno)
+        file_name.append(file_label)
+
+    """
+    for r in results:
+        CNO = len(r.boxes)
+        CNO_coor = np.empty([CNO, 2], dtype=int)
+        for j in range(CNO):
+            # w = r.boxes.xywh[j][2]
+            # h = r.boxes.xywh[j][3]
+            # area = (math.pi * w * h / 4) * 20 * 20 / (512 * 512)
+            # total_area += area
+            # bbox_img = r.orig_img
+            x = round(r.boxes.xywh[j][0].item())
+            y = round(r.boxes.xywh[j][1].item())
+
+            x1 = round(r.boxes.xyxy[j][0].item())
+            y1 = round(r.boxes.xyxy[j][1].item())
+            x2 = round(r.boxes.xyxy[j][2].item())
+            y2 = round(r.boxes.xyxy[j][3].item())
+
+            CNO_coor[j] = [x, y]
+            cv2.rectangle(r.orig_img, (x1, y1), (x2, y2), (0, 255, 0), 1)
+        im_array = r.orig_img
+        im = Image.fromarray(im_array[..., ::-1])
+
+    CNO_count = "CNO Count: " + str(CNO)
+    
+    test = []
+    for i in range(len(cno_image)):
+        test.append([cno_image[0], f"label {i}"])
+    """
+    data = {
+        "Files": file_name,
+        "CNO Count": cno_count,
+    }
+
+    # load data into a DataFrame object:
+    cno_df = pd.DataFrame(data)
+
+    return cno_df, cno_image
+
+def highlight_max(s, props=''):
+    return np.where(s == np.nanmax(s.values), props, '')
+
+def highlight_df(df, data: gr.SelectData):
+
+    styler = df.style.apply(lambda x: ['background: lightgreen'
+                               if x.Files == data.value["caption"]
+                               else None for i in x], axis=1)
+
+    # print("selected", data.value["caption"])
+    return data.value["caption"], styler
+
+def reset():
+    name_textbox = ""
+    gender_radio = None
+    age_slider = 0
+    fitzpatrick = 1
+    history = []
+    model_radio = "YOLOv8-M"
+    input_files = []
+    conf_slider = 0.2
+    iou_slider = 0.5
+    analysis_results = []
+    cno_gallery = []
+    test_label = ""
+
+    return name_textbox, gender_radio, age_slider, fitzpatrick, history, model_radio, input_files, conf_slider, iou_slider, analysis_results, cno_gallery, test_label
+
+
+with gr.Blocks(title="AFM AI Analysis", theme="default") as app:
+    with gr.Row():
+        with gr.Column():
+            # gr.Markdown("User Information")
+            with gr.Accordion("User Information", open=True):
+                name_textbox = gr.Textbox(label="Name")
+                with gr.Row():
+                    gender_radio = gr.Radio(["Male", "Female"], label="Gender", interactive=True, scale=1)
+                    age_slider = gr.Slider(minimum=0, maximum=100, step=1, value=0, label="Age", interactive=True, scale=2)
+                with gr.Group():
+                    fitzpatrick = gr.Slider(minimum=1, maximum=6, step=1, value=1, label="Fitzpatrick", interactive=True)
+                history = gr.Checkboxgroup(["Familial Disease", "Allergic Rhinitis", "Asthma"], label="Medical History", interactive=True)
+
+            input_files = gr.File(file_types=["image"], file_count="multiple", label="Upload Image")
+            # gr.Markdown("Model Configuration")
+            with gr.Accordion("Model Configuration", open=False):
+                model_radio = gr.Radio(["YOLOv8-N", "YOLOv8-S", "YOLOv8-M", "YOLOv8-L", "YOLOv8-X"], label="Model Selection", value="YOLOv8-M")
+                conf_slider = gr.Slider(minimum=0, maximum=1, value=0.2, label="Confidence threshold")
+                iou_slider = gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold")
+            with gr.Row():
+                analyze_btn = gr.Button("Analyze")
+                clear_btn = gr.Button("Reset")
+        with gr.Column():
+            analysis_results = gr.Dataframe(headers=["Files", "CNO Count"], interactive=False)
+            # cno_label = gr.Label(label="Analysis Results")
+            cno_gallery = gr.Gallery(label="Result", show_label=True, columns=3, object_fit="contain")
+            test_label = gr.Label(label="Analysis Results")
+            # cno_img = gr.Image(type="pil", label="Result")
+
+    analyze_btn.click(
+        fn=predict_image,
+        inputs=[name_textbox, model_radio, input_files, conf_slider, iou_slider],
+        outputs=[analysis_results, cno_gallery]
+    )
+
+    clear_btn.click(reset, outputs=[name_textbox, gender_radio, age_slider, fitzpatrick, history, model_radio,
+                                    input_files, conf_slider, iou_slider, analysis_results, cno_gallery, test_label])
+
+    cno_gallery.select(highlight_df, inputs=analysis_results, outputs=[test_label, analysis_results])
+
+
+"""
+iface = gr.Interface(
+    fn=predict_image,
+    inputs=[
+        gr.Textbox(label="User Name"),
+        gr.Radio(["YOLOv8-N", "YOLOv8-S", "YOLOv8-M", "YOLOv8-L", "YOLOv8-X"], value="YOLOv8-M"),
+        # gr.Image(type="filepath", label="Upload Image"),
+        gr.File(file_types=["image"], file_count="multiple", label="Upload Image"),
+        gr.Slider(minimum=0, maximum=1, value=0.2, label="Confidence threshold"),
+        gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold")
+    ],
+    outputs=[gr.Label(label="Analysis Results"), gr.Image(type="pil", label="Result")],
+    title="AFM AI Analysis",
+    description="Upload images for inference. The YOLOv8-M model is used by default.",
+    theme=gr.themes.Default()
+)
+"""
+
+if __name__ == '__main__':
+    # iface.launch()
+    app.launch(auth=('user', 'admin'), auth_message="Enter your username and password")

web_test.py

@@ -0,0 +1,93 @@
+import os
+import cv2
+import PIL.Image as Image
+import gradio as gr
+import numpy as np
+import math
+from pathlib import Path
+from ultralytics import ASSETS, YOLO
+
+DIR_NAME = Path(os.path.dirname(__file__))
+DETECTION_MODEL_n = os.path.join(DIR_NAME, 'models', 'YOLOv8-N_CNO_Detection.pt')
+DETECTION_MODEL_s = os.path.join(DIR_NAME, 'models', 'YOLOv8-S_CNO_Detection.pt')
+DETECTION_MODEL_m = os.path.join(DIR_NAME, 'models', 'YOLOv8-M_CNO_Detection.pt')
+DETECTION_MODEL_l = os.path.join(DIR_NAME, 'models', 'YOLOv8-L_CNO_Detection.pt')
+DETECTION_MODEL_x = os.path.join(DIR_NAME, 'models', 'YOLOv8-X_CNO_Detection.pt')
+
+# MODEL = os.path.join(DIR_NAME, 'models', 'YOLOv8-M_CNO_Detection.pt')
+# model = YOLO(MODEL)
+
+
+def predict_image(name, model, img, conf_threshold, iou_threshold):
+    """Predicts and plots labeled objects in an image using YOLOv8 model with adjustable confidence and IOU thresholds."""
+    gr.Info("Starting process")
+    # gr.Warning("Name is empty")
+    if name == "":
+        gr.Warning("Name is empty")
+
+    if model == 'YOLOv8-N':
+        CNO_model = YOLO(DETECTION_MODEL_n)
+    elif model == 'YOLOv8-S':
+        CNO_model = YOLO(DETECTION_MODEL_s)
+    elif model == 'YOLOv8-M':
+        CNO_model = YOLO(DETECTION_MODEL_m)
+    elif model == 'YOLOv8-L':
+        CNO_model = YOLO(DETECTION_MODEL_l)
+    else:
+        CNO_model = YOLO(DETECTION_MODEL_x)
+
+    results = CNO_model.predict(
+        source=img,
+        conf=conf_threshold,
+        iou=iou_threshold,
+        show_labels=False,
+        show_conf=False,
+        imgsz=512,
+        max_det=1200
+    )
+
+    for r in results:
+        CNO = len(r.boxes)
+        CNO_coor = np.empty([CNO, 2], dtype=int)
+        for j in range(CNO):
+            # w = r.boxes.xywh[j][2]
+            # h = r.boxes.xywh[j][3]
+            # area = (math.pi * w * h / 4) * 20 * 20 / (512 * 512)
+            # total_area += area
+            # bbox_img = r.orig_img
+            x = round(r.boxes.xywh[j][0].item())
+            y = round(r.boxes.xywh[j][1].item())
+
+            x1 = round(r.boxes.xyxy[j][0].item())
+            y1 = round(r.boxes.xyxy[j][1].item())
+            x2 = round(r.boxes.xyxy[j][2].item())
+            y2 = round(r.boxes.xyxy[j][3].item())
+
+            CNO_coor[j] = [x, y]
+            cv2.rectangle(r.orig_img, (x1, y1), (x2, y2), (0, 255, 0), 1)
+        im_array = r.orig_img
+        im = Image.fromarray(im_array[..., ::-1])
+
+    CNO_count = "CNO Count: " + str(CNO)
+
+    return CNO_count, im
+
+
+iface = gr.Interface(
+    fn=predict_image,
+    inputs=[
+        gr.Textbox(label="User Name"),
+        gr.Radio(["YOLOv8-N", "YOLOv8-S", "YOLOv8-M", "YOLOv8-L", "YOLOv8-X"], value="YOLOv8-M"),
+        # gr.Image(type="filepath", label="Upload Image"),
+        gr.File(file_types=["image"], file_count="multiple", label="Upload Image"),
+        gr.Slider(minimum=0, maximum=1, value=0.2, label="Confidence threshold"),
+        gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold")
+    ],
+    outputs=[gr.Label(label="Analysis Results"), gr.Image(type="pil", label="Result")],
+    title="AFM AI Analysis",
+    description="Upload images for inference. The YOLOv8-M model is used by default.",
+    theme=gr.themes.Default()
+)
+
+if __name__ == '__main__':
+    iface.launch()