CNN based Rapid Cancer Diagnosis

AI-Powered Real-Time Histopathological Image Classification for Rapid Cancer Diagnosis

Revolutionizing Cancer Diagnosis with AI-Powered Histopathology

The timely and accurate diagnosis of cancer is paramount for effective treatment and improved patient outcomes. Traditional histopathological diagnosis, while precise, is often time-consuming, labor-intensive, and subject to inter-observer variability. This article presents a real-time AI-powered system for automated classification of histopathological images, leveraging advanced Convolutional Neural Networks (CNNs). Inspired by efficient waste segregation, this project adapts similar principles to rapidly identify cancerous cells within biopsy samples. This system aims to significantly reduce diagnosis time, enhance diagnostic accuracy, and provide crucial support to pathologists, thereby revolutionizing the cancer diagnosis workflow in medical science.

1. Introduction

Cancer remains a leading cause of mortality worldwide. Early detection and precise characterization of cancerous tissue are critical for guiding therapeutic decisions. Histopathology, the microscopic examination of tissue biopsies, is the gold standard for cancer diagnosis. However, the manual analysis of countless tissue slides is a demanding task, requiring extensive training and experience. Pathologists face immense workloads, leading to potential delays and the possibility of human error, especially in complex cases.

The advent of Artificial Intelligence (AI), particularly deep learning, offers an unprecedented opportunity to augment human capabilities in medical diagnostics. Convolutional Neural Networks (CNNs) have shown remarkable success in image recognition tasks, making them ideal candidates for analyzing complex medical images. This project proposes a real-time system that utilizes CNNs to classify histopathological images into cancerous and non-cancerous categories, or even specific cancer subtypes. By automating this initial screening, pathologists can prioritize their focus on challenging cases, leading to faster and more consistent diagnoses.

2. Project Objective

The primary objective of this project is to develop and deploy an efficient, accurate, and real-time AI system for the classification of histopathological images.

The key goals are:

• Accelerate Diagnosis: Significantly reduce the time required for initial screening of biopsy slides.
• Enhance Accuracy: Improve the consistency and accuracy of cancer diagnosis by providing an objective, AI-driven assessment.
• Reduce Workload: Assist pathologists by automating routine classification tasks, allowing them to concentrate on complex cases.
• Facilitate Research: Create a robust framework that can be extended for research into various cancer types and their histopathological features.
• Real-time Implementation: Design the system for rapid inference, enabling near-instantaneous classification of new images.

3. Use Case in Medical Science: Breast Cancer Histopathology

Breast cancer is one of the most common cancers among women globally. Histopathological analysis of breast biopsies is crucial for diagnosis and grading. This project focuses on a binary classification task: differentiating between benign and malignant breast tissue images. This is a highly relevant and impactful use case because:

• High Prevalence: Breast cancer affects millions, making efficient diagnosis a critical need.
• Clear Classification: Benign vs. Malignant is a well-defined diagnostic task in histopathology.
• Impact on Treatment: Accurate and swift diagnosis directly impacts treatment pathways (e.g., surgery, chemotherapy, radiation).
• Scalability: The principles can be extended to multi-class classification (e.g., different grades of malignancy, specific tumor types).

In a real-time scenario, this system could be integrated into digital pathology workflows. As a pathologist scans a new slide, the AI system could provide immediate feedback on suspicious regions, flagging them for closer human inspection. This acts as a powerful “second opinion” or an initial filter.

# Assuming the zip file is named 'BreaKHis_v1.zip' and uploaded to Colab's current directory
!unzip BreaKHis_v1.zip -d BreaKHis_v1_extracted

data_root_dir = 'BreaKHis_v1_extracted'

4. Data Understanding

For histopathological image classification, publicly available datasets are crucial for training robust models. One very relevant and widely used dataset is:

BreakHis Dataset (Breast Cancer Histopathological Database)

• Description: The BreakHis dataset consists of 7,909 microscopic images of breast tumor biopsies, collected from 82 patients. It’s organized into two main categories: Benign and Malignant.
  • Benign Subtypes: Adenosis, Fibroadenoma, Phyllodes Tumor, Tubular Adenoma.
  • Malignant Subtypes: Ductal Carcinoma, Lobular Carcinoma, Mucinous Carcinoma, Papillary Carcinoma.
• Magnification Levels: Images are available at 40x, 100x, 200x, and 400x magnification. This allows for training models capable of handling different resolutions, mirroring real-world diagnostic scenarios where pathologists examine slides at varying magnifications.
• Image Format: Typically in PNG format.
• Challenges:
  • Image Heterogeneity: Variations in staining, lighting, and tissue preparation can introduce noise.
  • Imbalance: Some subtypes or magnification levels might have fewer samples.
  • Subtle Differences: Distinguishing benign from malignant often relies on subtle cellular and architectural features.

For this project, we will focus on the binary classification (Benign vs. Malignant) across all magnification levels, treating each image as an independent sample.

5. Technical Implementation: Code Structure and Explanation

The Python code will follow a structured machine learning pipeline: data loading, preprocessing, visualization, model building, training, and evaluation. We will use TensorFlow and Keras for deep learning, scikit-learn for data splitting and metrics, and Pillow, NumPy, Matplotlib, Seaborn for image handling and visualization.

5.1. Import Necessary Libraries

This section imports all required libraries. Version recommendations are provided to ensure reproducibility.

Python

# Import essential libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import zipfile # For handling compressed datasets if needed
from PIL import Image # For image loading and manipulation
from sklearn.model_selection import train_test_split # For splitting data
from sklearn.preprocessing import LabelEncoder # For encoding categorical labels
from sklearn.metrics import classification_report, confusion_matrix, accuracy_score, precision_score, recall_score, f1_score # For model evaluation

import tensorflow as tf
from tensorflow.keras.models import Sequential, load_model # For building and loading models
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization # Core CNN layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator # For data augmentation
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint # Callbacks for training
from tensorflow.keras.optimizers import Adam # Optimizer
from tensorflow.keras.utils import to_categorical # For one-hot encoding labels

# Set random seed for reproducibility
np.random.seed(42)
tf.random.set_seed(42)

print(f"TensorFlow Version: {tf.__version__}")
print(f"Keras Version: {tf.keras.__version__}") # Keras is now integrated into TensorFlow

5.2. Data Loading and Preprocessing

This crucial step involves loading the images, extracting labels, and preparing them for the CNN. Histopathological images can be large; therefore, careful resizing and normalization are essential.

Python

# --- 1. Load the data ---
# In a real-world scenario, you might download from Kaggle or other sources.
# For demonstration, assume the dataset is already extracted or a path to it is provided.

# Define the base directory for the BreakHis dataset
# IMPORTANT: Adjust this path to where your BreakHis dataset is located or extracted.
# Example structure:
# breakhis_dataset/
# ├── B/ (Benign)
# │   ├── SOF/ (Subtype: Fibroadenoma)
# │   │   ├── For/ (Magnification: 40x)
# │   │   │   ├── ... .png
# ├── M/ (Malignant)
# │   ├── IDC/ (Subtype: Invasive Ductal Carcinoma)
# │   │   ├── For/ (Magnification: 40x)
# │   │   │   ├── ... .png

data_root_dir = 'BreaKHis_v1' # Example: Assuming 'BreaKHis_v1' is the extracted root folder

# Function to load images and labels
def load_breakhis_data(root_dir, target_size=(128, 128)):
    """
    Loads images from the BreaKHis dataset, classifying them as 'benign' or 'malignant'.
    Resizes images to target_size.

    Args:
        root_dir (str): Path to the root directory of the BreaKHis dataset (e.g., 'BreaKHis_v1').
        target_size (tuple): Desired (width, height) to resize images.

    Returns:
        tuple: A tuple containing (numpy_images_array, list_labels).
    """
    images = []
    labels = []
    print(f"Loading data from: {root_dir}")

    # The BreaKHis structure is typically root_dir/[B,M]/[subtypes]/[magnification_folders]/images
    for cancer_type_dir in ['B', 'M']: # 'B' for Benign, 'M' for Malignant
        type_path = os.path.join(root_dir, cancer_type_dir)
        if not os.path.exists(type_path):
            print(f"Warning: Directory {type_path} not found. Skipping.")
            continue

        for subtype_dir in os.listdir(type_path):
            subtype_path = os.path.join(type_path, subtype_dir)
            if not os.path.isdir(subtype_path):
                continue # Skip files, only process directories

            for mag_level_dir in os.listdir(subtype_path):
                mag_path = os.path.join(subtype_path, mag_level_dir)
                if not os.path.isdir(mag_path):
                    continue

                for patient_dir in os.listdir(mag_path): # Individual patient folders
                    patient_path = os.path.join(mag_path, patient_dir)
                    if not os.path.isdir(patient_path):
                        continue

                    for image_file in os.listdir(patient_path):
                        if image_file.lower().endswith(('.png', '.jpg', '.jpeg')):
                            image_path = os.path.join(patient_path, image_file)
                            try:
                                img = Image.open(image_path).convert('RGB')
                                img = img.resize(target_size, Image.LANCZOS) # High-quality downsampling filter
                                images.append(np.array(img))
                                # Assign 'Benign' or 'Malignant' label
                                labels.append('Benign' if cancer_type_dir == 'B' else 'Malignant')
                            except Exception as e:
                                print(f"Could not load image {image_path}: {e}")

    return np.array(images), labels

# Define target image size for CNN input
TARGET_IMG_SIZE = (128, 128) # Height x Width for breast cancer images

# Load images and labels
# NOTE: This might take a few minutes depending on the dataset size and system.
# Ensure the 'BreaKHis_v1' directory is correctly set up.
all_images, all_labels = load_breakhis_data(data_root_dir, TARGET_IMG_SIZE)

print(f"Total images loaded: {len(all_images)}")
print(f"Total labels loaded: {len(all_labels)}")

# Normalize pixel values to [0, 1]
X = all_images.astype('float32') / 255.0

# Encode labels
label_encoder = LabelEncoder()
y_encoded = label_encoder.fit_transform(all_labels) # 0 for Benign, 1 for Malignant
num_classes = len(label_encoder.classes_)
class_names = label_encoder.classes_ # ['Benign', 'Malignant']

# One-hot encode labels for categorical crossentropy
y = to_categorical(y_encoded, num_classes=num_classes)

print(f"Shape of preprocessed images (X): {X.shape}")
print(f"Shape of one-hot encoded labels (y): {y.shape}")
print(f"Class names: {class_names}")

5.3. Data Visualization

Understanding the data distribution and seeing sample images helps in grasping the dataset’s characteristics.

Python

# --- 2.2 Data Visualisation ---
# 2.2.1 Create a bar plot to display the class distribution
plt.figure(figsize=(8, 5))
sns.countplot(x=all_labels, palette='coolwarm', order=pd.Series(all_labels).value_counts().index)
plt.title('Distribution of Breast Cancer Histopathology Classes')
plt.xlabel('Cancer Type')
plt.ylabel('Number of Images')
plt.grid(axis='y', linestyle='--', alpha=0.7)
plt.show()

print("\nClass distribution details:")
print(pd.Series(all_labels).value_counts())

# 2.2.2 Visualise some sample images
def plot_sample_images(images, labels, class_names, num_samples=8):
    """
    Plots sample images from the dataset with their corresponding labels.
    Ensures a mix of classes if possible.
    """
    plt.figure(figsize=(15, 8))
    unique_labels_indices = [np.where(np.argmax(labels, axis=1) == i)[0] for i in range(len(class_names))]
    
    selected_indices = []
    # Try to pick at least one from each class
    for indices_for_class in unique_labels_indices:
        if len(indices_for_class) > 0:
            selected_indices.append(np.random.choice(indices_for_class))

    # Fill up to num_samples with random images if needed
    while len(selected_indices) < num_samples:
        rand_idx = np.random.randint(0, len(images))
        if rand_idx not in selected_indices:
            selected_indices.append(rand_idx)

    for i, idx in enumerate(selected_indices[:num_samples]):
        ax = plt.subplot(2, num_samples // 2, i + 1)
        plt.imshow(images[idx])
        plt.title(f"{class_names[np.argmax(labels[idx])]}")
        plt.axis("off")
    plt.tight_layout()
    plt.show()

print("\nSample Images from Dataset:")
plot_sample_images(X, y, class_names, num_samples=10)

5.4. Data Splitting

Splitting the dataset into training and validation sets is crucial for evaluating model generalization. Stratified splitting ensures that class distribution is maintained.

Python

# --- 2.4 Data Splitting ---
# 2.4.1 Split the dataset into training and validation sets
# Using 80% for training and 20% for validation
# Stratify by 'y_encoded' to maintain class distribution in both sets, important for imbalanced datasets
X_train, X_val, y_train, y_val = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y_encoded
)

print(f"Shape of X_train: {X_train.shape}")
print(f"Shape of X_val: {X_val.shape}")
print(f"Shape of y_train: {y_train.shape}")
print(f"Shape of y_val: {y_val.shape}")

# Verify the class distribution in training and validation sets
print("\nTraining set class distribution (encoded):")
print(pd.Series(np.argmax(y_train, axis=1)).value_counts().sort_index())

print("\nValidation set class distribution (encoded):")
print(pd.Series(np.argmax(y_val, axis=1)).value_counts().sort_index())

5.5. Model Building and Training (Baseline Model)

Here, we construct a CNN model suitable for image classification. A robust architecture includes convolutional layers, pooling layers, batch normalization, and dropout for regularization.

Python

# --- 3. Model Building and Evaluation ---
# 3.1 Model building and training
# 3.1.1 Build and compile the model (Baseline Model without augmentation)

def build_cnn_model(input_shape, num_classes):
    """
    Builds a Sequential CNN model for image classification.
    Incorporates Conv2D, MaxPooling2D, BatchNormalization, and Dropout layers.

    Args:
        input_shape (tuple): Shape of the input images (height, width, channels).
        num_classes (int): Number of output classes.

    Returns:
        tf.keras.Model: Compiled Keras Sequential model.
    """
    model = Sequential([
        # First Convolutional Block
        Conv2D(32, (3, 3), activation='relu', input_shape=input_shape, padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),
        Dropout(0.25),

        # Second Convolutional Block
        Conv2D(64, (3, 3), activation='relu', padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),
        Dropout(0.25),

        # Third Convolutional Block
        Conv2D(128, (3, 3), activation='relu', padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),
        Dropout(0.35), # Slightly higher dropout for more complex features

        # Flatten the output for the fully connected layers
        Flatten(),

        # Fully Connected Layers
        Dense(256, activation='relu'),
        BatchNormalization(),
        Dropout(0.5), # Significant dropout for the dense layer to prevent overfitting

        # Output layer
        Dense(num_classes, activation='softmax')
    ])

    # Compile the model
    optimizer = Adam(learning_rate=0.001) # Using Adam optimizer with a default learning rate
    model.compile(optimizer=optimizer,
                  loss='categorical_crossentropy',
                  metrics=['accuracy'])
    return model

input_shape = (TARGET_IMG_SIZE[0], TARGET_IMG_SIZE[1], 3)
baseline_model = build_cnn_model(input_shape, num_classes)
print("Baseline Model Summary:")
baseline_model.summary()

# 3.1.2 Train the model (Baseline Model)
# Define callbacks
early_stopping = EarlyStopping(monitor='val_accuracy', patience=15, restore_best_weights=True, verbose=1)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=7, min_lr=0.00001, verbose=1)
model_checkpoint = ModelCheckpoint('best_baseline_model.keras', monitor='val_accuracy', save_best_only=True, mode='max', verbose=1)

print("\n--- Training Baseline Model ---")
history_baseline = baseline_model.fit(
    X_train, y_train,
    epochs=100, # Increased epochs as EarlyStopping will manage it
    batch_size=32,
    validation_data=(X_val, y_val),
    callbacks=[early_stopping, reduce_lr, model_checkpoint],
    verbose=1
)

# Plot training history for baseline model
def plot_training_history(history, title_suffix=""):
    plt.figure(figsize=(12, 5))

    # Plot accuracy
    plt.subplot(1, 2, 1)
    plt.plot(history.history['accuracy'], label='Train Accuracy')
    plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
    plt.title(f'Model Accuracy {title_suffix}')
    plt.xlabel('Epoch')
    plt.ylabel('Accuracy')
    plt.legend()
    plt.grid(True)

    # Plot loss
    plt.subplot(1, 2, 2)
    plt.plot(history.history['loss'], label='Train Loss')
    plt.plot(history.history['val_loss'], label='Validation Loss')
    plt.title(f'Model Loss {title_suffix}')
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.legend()
    plt.grid(True)
    plt.tight_layout()
    plt.show()

plot_training_history(history_baseline, "(Baseline Model)")

5.6. Model Testing and Evaluation (Baseline Model)

Rigorous evaluation using various metrics is essential to understand the model’s performance beyond just accuracy.

Python

# --- 3.2 Model Testing and Evaluation (Baseline Model) ---
# 3.2.1 Evaluate the model on validation dataset. Derive appropriate metrics.

print("\n--- Evaluating Baseline Model on Validation Set ---")
baseline_loss, baseline_accuracy = baseline_model.evaluate(X_val, y_val, verbose=1)

print(f"\nBaseline Model Validation Loss: {baseline_loss:.4f}")
print(f"Baseline Model Validation Accuracy: {baseline_accuracy:.4f}")

# Get predictions
y_pred_probs_baseline = baseline_model.predict(X_val)
y_pred_baseline = np.argmax(y_pred_probs_baseline, axis=1) # Predicted classes
y_true_val = np.argmax(y_val, axis=1) # True classes

# Classification Report
print("\nClassification Report (Baseline Model):")
print(classification_report(y_true_val, y_pred_baseline, target_names=class_names))

# Confusion Matrix
conf_matrix_baseline = confusion_matrix(y_true_val, y_pred_baseline)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_baseline, annot=True, fmt='d', cmap='Blues',
            xticklabels=class_names, yticklabels=class_names)
plt.title('Confusion Matrix (Baseline Model)')
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.show()

# Calculate and print additional metrics
print(f"Precision (Macro Avg): {precision_score(y_true_val, y_pred_baseline, average='macro'):.4f}")
print(f"Recall (Macro Avg): {recall_score(y_true_val, y_pred_baseline, average='macro'):.4f}")
print(f"F1-Score (Macro Avg): {f1_score(y_true_val, y_pred_baseline, average='macro'):.4f}")

5.7. Data Augmentation and Augmented Model Training

Data augmentation artificially expands the training dataset, making the model more robust to variations in real-world images and reducing overfitting.

Python

# --- 4. Data Augmentation ---
# 4.1 Create a Data Augmentation Pipeline
# 4.1.1 Define augmentation steps for the datasets.

# Create an ImageDataGenerator for data augmentation
train_datagen = ImageDataGenerator(
    rotation_range=20,          # Random rotation from -20 to +20 degrees
    zoom_range=0.15,            # Random zoom range
    width_shift_range=0.2,      # Random horizontal shift
    height_shift_range=0.2,     # Random vertical shift
    shear_range=0.15,           # Shear intensity
    horizontal_flip=True,       # Randomly flip inputs horizontally
    vertical_flip=False,        # Vertical flip usually not beneficial for histopathology
    fill_mode='nearest',        # Strategy for filling in new pixels
    brightness_range=[0.8, 1.2] # Randomly adjust brightness
)

# For validation data, we only rescale/normalize, no augmentation
val_datagen = ImageDataGenerator() # X_val is already normalized, so no rescale needed

# Create augmented training and validation data generators
augmented_train_generator = train_datagen.flow(X_train, y_train, batch_size=32, shuffle=True)
validation_generator = val_datagen.flow(X_val, y_val, batch_size=32, shuffle=False)

print("Data augmentation pipeline defined and generators created.")

# 4.1.2 Train the model on the new augmented dataset.
# Re-build the model to ensure fresh weights for fair comparison with augmentation.
augmented_model = build_cnn_model(input_shape, num_classes)
print("\nAugmented Model Summary:")
augmented_model.summary()

# Define callbacks for augmented training
early_stopping_aug = EarlyStopping(monitor='val_accuracy', patience=20, restore_best_weights=True, verbose=1)
reduce_lr_aug = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, min_lr=0.000001, verbose=1)
model_checkpoint_aug = ModelCheckpoint('best_augmented_model.keras', monitor='val_accuracy', save_best_only=True, mode='max', verbose=1)

print("\n--- Training Augmented Model ---")
history_augmented = augmented_model.fit(
    augmented_train_generator,
    steps_per_epoch=len(X_train) // 32, # To cover all training samples roughly once per epoch
    epochs=150, # Increased epochs, augmentation often needs more
    validation_data=validation_generator,
    validation_steps=len(X_val) // 32, # To cover all validation samples
    callbacks=[early_stopping_aug, reduce_lr_aug, model_checkpoint_aug],
    verbose=1
)

plot_training_history(history_augmented, "(Augmented Model)")

# --- 3.2 Model Testing and Evaluation (Augmented Model) ---
print("\n--- Evaluating Augmented Model on Validation Set ---")
augmented_loss, augmented_accuracy = augmented_model.evaluate(X_val, y_val, verbose=1)

print(f"\nAugmented Model Validation Loss: {augmented_loss:.4f}")
print(f"Augmented Model Validation Accuracy: {augmented_accuracy:.4f}")

# Get predictions for augmented model
y_pred_probs_augmented = augmented_model.predict(X_val)
y_pred_augmented = np.argmax(y_pred_probs_augmented, axis=1)

# Classification Report (Augmented Model)
print("\nClassification Report (Augmented Model):")
print(classification_report(y_true_val, y_pred_augmented, target_names=class_names))

# Confusion Matrix (Augmented Model)
conf_matrix_augmented = confusion_matrix(y_true_val, y_pred_augmented)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_augmented, annot=True, fmt='d', cmap='Blues',
            xticklabels=class_names, yticklabels=class_names)
plt.title('Confusion Matrix (Augmented Model)')
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.show()

print(f"Precision (Macro Avg, Augmented): {precision_score(y_true_val, y_pred_augmented, average='macro'):.4f}")
print(f"Recall (Macro Avg, Augmented): {recall_score(y_true_val, y_pred_augmented, average='macro'):.4f}")
print(f"F1-Score (Macro Avg, Augmented): {f1_score(y_true_val, y_pred_augmented, average='macro'):.4f}")

5.8. Real-time Inference Simulation

This section demonstrates how the trained model would perform in a real-time scenario, classifying a single image.

Python

# --- Real-time Inference Simulation ---

def predict_single_image(model, image_path, target_size, class_names):
    """
    Loads, preprocesses, and predicts the class of a single histopathological image.

    Args:
        model (tf.keras.Model): The trained CNN model.
        image_path (str): Path to the image file.
        target_size (tuple): The (width, height) the model expects.
        class_names (list): List of class names corresponding to model output.

    Returns:
        tuple: (predicted_class_name, prediction_probabilities_array)
    """
    try:
        img = Image.open(image_path).convert('RGB')
        img = img.resize(target_size, Image.LANCZOS)
        img_array = np.array(img).astype('float32') / 255.0
        
        # Add batch dimension: (height, width, channels) -> (1, height, width, channels)
        img_array = np.expand_dims(img_array, axis=0)

        predictions = model.predict(img_array)
        predicted_class_idx = np.argmax(predictions)
        predicted_class_name = class_names[predicted_class_idx]
        
        return predicted_class_name, predictions[0] # Return probabilities for the single image

    except Exception as e:
        print(f"Error during prediction for {image_path}: {e}")
        return None, None

# Example of how to use for real-time inference (you'll need actual image paths)
# For demonstration, let's pick a random image from the validation set
# In a real system, this would be a new incoming biopsy image.

print("\n--- Simulating Real-time Inference ---")

# Load the best augmented model for inference
try:
    final_model = load_model('best_augmented_model.keras')
    print("Loaded best augmented model for inference.")
except Exception as e:
    print(f"Could not load 'best_augmented_model.keras'. Using the last trained augmented model. Error: {e}")
    final_model = augmented_model # Fallback to the last trained model if checkpoint fails

# Select a random image from the validation set for demonstration
random_idx = np.random.randint(0, len(X_val))
sample_image_array = X_val[random_idx]
true_label_idx = np.argmax(y_val[random_idx])
true_label_name = class_names[true_label_idx]

# Create a dummy image path for the function (as it expects a path)
# In a real application, you'd save the image to a temp path or pass array directly
# For this simulation, we'll bypass saving and directly pass the array.
# Let's create a dummy function to simulate loading from path for completeness.
def predict_from_array(model, img_array, class_names):
    img_array_expanded = np.expand_dims(img_array, axis=0)
    predictions = model.predict(img_array_expanded)
    predicted_class_idx = np.argmax(predictions)
    predicted_class_name = class_names[predicted_class_idx]
    return predicted_class_name, predictions[0]

predicted_class, probabilities = predict_from_array(final_model, sample_image_array, class_names)

print(f"\n--- Prediction for a Sample Image ---")
print(f"True Label: {true_label_name}")
print(f"Predicted Label: {predicted_class}")
print(f"Prediction Probabilities: {probabilities}")

# Visualize the sample image and its prediction
plt.figure(figsize=(6, 6))
plt.imshow(sample_image_array)
plt.title(f"True: {true_label_name}\nPredicted: {predicted_class} (Conf: {probabilities[np.argmax(probabilities)]:.2f})")
plt.axis('off')
plt.show()

# Example: If you had an actual image file somewhere:
# test_image_path = "path/to/a/new_biopsy_image.png"
# predicted_class_file, probabilities_file = predict_single_image(final_model, test_image_path, TARGET_IMG_SIZE, class_names)
# if predicted_class_file:
#     print(f"\nPrediction for {test_image_path}: {predicted_class_file} with probabilities {probabilities_file}")

6. Real-time Project Architecture for Clinical Deployment

For a true “real-time” system in a clinical setting, the architecture would involve:

1. Digital Pathology Scanner: Acquires high-resolution whole slide images (WSI) from biopsy slides.
2. Image Segmentation/Patching: WSIs are too large for direct CNN input. They are typically divided into smaller, overlapping “patches” or tiles.
3. Real-time Inference Engine (Edge/Cloud):
   • Edge Device: A powerful workstation with a GPU located in the pathology lab. This minimizes latency and ensures data privacy.
   • Cloud Service: For larger hospitals or research centers, models could be deployed on cloud platforms (e.g., AWS SageMaker, Google AI Platform, Azure ML) offering scalability and managed services.
   • Model Loading: The pre-trained best_augmented_model.keras would be loaded into memory.
   • Patch Processing: Each patch is fed to the CNN model.
   • Prediction Aggregation: Predictions from individual patches are aggregated to provide a diagnosis for the entire slide (e.g., majority voting, heatmaps indicating cancerous regions).
4. User Interface (UI): A custom application for pathologists to upload slides, view AI predictions (e.g., heatmaps overlaid on the WSI), and provide their final diagnosis.
5. Database/Reporting: Store predictions, pathologist’s annotations, and final diagnoses for record-keeping and auditing.

Challenges in Real-time Deployment:

• Whole Slide Image (WSI) Handling: WSIs can be tens of gigabytes, making loading, processing, and visualization challenging. Efficient tiling and multi-resolution imaging are critical.
• Computational Resources: Real-time processing of high-throughput data requires significant GPU power.
• Clinical Integration: Seamless integration with existing laboratory information systems (LIS) and hospital information systems (HIS).
• Regulatory Approval: Medical devices using AI require stringent validation and regulatory approval (e.g., FDA, CE Mark).
• Interpretability: Pathologists need to understand why the AI made a certain prediction, demanding explainable AI (XAI) techniques.

7. Conclusions and Future Work

This project demonstrates a robust framework for applying CNNs to histopathological image classification, specifically for breast cancer diagnosis.

• Outcomes and Insights Gained:
  • Feasibility of CNNs: CNNs are highly effective in distinguishing between benign and malignant breast cancer tissues based on microscopic images.
  • Importance of Preprocessing: Proper image resizing and normalization are crucial for model performance and efficiency.
  • Impact of Data Augmentation: Data augmentation significantly improves the model’s generalization capabilities, making it more resilient to real-world image variations and helping mitigate overfitting. The augmented model is expected to outperform the baseline model, showing higher validation accuracy and better F1-scores across classes.
  • Value of Callbacks: Early Stopping and ReduceLROnPlateau are essential for efficient training, preventing overfitting and optimizing learning.
  • Comprehensive Evaluation: Beyond simple accuracy, using a classification report (precision, recall, F1-score) and a confusion matrix provides a detailed understanding of the model’s strengths and weaknesses for each class, which is critical in medical applications. (Upon execution, we’d report specific numbers here, e.g., “The augmented model achieved a validation accuracy of X% and an F1-score of Y for the malignant class, indicating its strong capability in detecting cancerous cells.”)
• Future Enhancements:
  • Transfer Learning: Utilize pre-trained models (e.g., ResNet, Inception, VGG) on large image datasets (ImageNet) and fine-tune them on histopathology data. This can lead to superior performance with less training data.
  • Advanced Architectures: Explore more sophisticated CNN architectures or vision transformers for improved feature extraction.
  • Multi-magnification Fusion: Develop models that can effectively combine information from different magnification levels (e.g., 40x, 100x, 200x, 400x) for a more comprehensive diagnosis.
  • Localization and Segmentation: Beyond classification, implement object detection (e.g., YOLO, Faster R-CNN) or semantic segmentation (e.g., U-Net) to pinpoint specific cancerous regions or individual cells on the slide.
  • Explainable AI (XAI): Integrate techniques like Grad-CAM or SHAP to visualize which parts of the image the model focuses on for its predictions, increasing trust and interpretability for pathologists.
  • Integration with Clinical Workflow: Develop a robust, user-friendly interface and backend for seamless integration into digital pathology systems.
  • Larger and Diverse Datasets: Train on more extensive and diverse datasets, ideally from multiple institutions, to ensure robustness across different patient populations and lab protocols.
  • Handling Imbalance: For datasets with severe class imbalance (e.g., rare cancer subtypes), implement advanced techniques like focal loss, weighted loss functions, or sophisticated resampling methods (SMOTE, ADASYN) at the data generator level.

This AI-powered histopathological image classification system represents a significant step towards enabling faster, more accurate, and ultimately life-saving cancer diagnoses, paving the way for the next generation of precision medicine.