Skin Lesion AI Detection

Phase 1: Data Engineering & Medical Dataset Analysis

The foundation of any successful AI project lies in understanding and preparing your data. I used the HAM10k dataset (10,015 dermatoscopic images, 7 lesion classes, patients aged 0–85). Perfect, but with some significant challenges:

• Severe class imbalance: Most lesions were benign, creating a skewed dataset that would bias my model.
• Strategic focus: To maintain balance, I focused on the critical binary classification: Melanoma vs Benign Keratosis.
• Preprocessing pipeline: Scaled down images, normalised pixel values, and stored everything in a Pandas dataframe.

• Age distribution analysis: Created histograms showing negatively skewed distribution (as expected in medical datasets).
• Diagnosis breakdown: Built pie charts revealing the clear class imbalance that needed addressing.
• Training monitoring: Developed real-time graphs tracking accuracy vs loss over training epochs.

Technical stack: NumPy for numerical operations, Pandas for data manipulation, Pillow for image preprocessing, and Plotly + Matplotlib for interactive visualisation. This gave me a comprehensive understanding of the data before I even touched neural networks.

Phase 2: Deep Learning Architecture & CNN Development

This was the core of the project: training a deep learning model with TensorFlow/Keras to perform medical image classification at professional standards.

• Image preprocessing: Implemented robust image scaling, normalisation, and class balancing techniques.
• Architecture design: Built custom CNN with convolution → pooling → dense layers → sigmoid output structure.
• Class imbalance handling: Added class weights to prevent bias toward majority classes.
• Activation functions: Implemented ReLU + Softmax for optimal gradient flow and classification.
• Loss optimization: Used binary cross-entropy with careful hyperparameter tuning.
• Training optimization: Normalised inputs and implemented callbacks for automated training tuning.

• Initial models: Early attempts hovered at 10-15% accuracy for 7 classes (essentially random guessing).
• Iterative improvement: Enhanced preprocessing (eliminated aggressive downscaling), implemented better tuning strategies, and added data augmentation techniques.
• Final performance: Achieved 93% training accuracy and ~75% generalised accuracy on unseen test data.
• Statistical validation: Calculated the 95% confidence interval for real accuracy: Between 71%-81%.

Key achievement: The model wasn't just memorising training data — it actually generalised to new samples, which is the gold standard in AI and essential for medical applications.

Phase 3: Full-Stack System Architecture & Deployment

The CNN was only half the battle. I wanted to make it practically usable in real healthcare scenarios, not just a research prototype.

• Frontend development: Built responsive HTML, CSS, JavaScript interface with interactive graphs and image upload functionality.
• Backend infrastructure: Developed Flask server (Python) to process uploads and orchestrate the entire system.
• Database integration: Implemented MySQL (via Python mysql.connector) storing comprehensive patient metadata (age, gender, diagnosis).

• API architecture: Built RESTful API allowing JavaScript to send user input as JSON → Flask → database seamlessly.
• Dynamic visualisations: IIS displayed real-time plots based on user inputs (age distributions, gender breakdowns, diagnostic patterns).
• AI integration: Flask endpoint enabled users to upload lesion images → process through saved .keras CNN model → return live predictions.
• Real-time processing: Implemented efficient image preprocessing pipeline for instant AI classification results.

Transformation achieved: This turned the project from "just a neural net" into a working diagnostic tool that could run on a real web application, demonstrating production-ready AI deployment capabilities.

Phase 4: Model Evaluation, System Validation & Production Readiness

With the complete system built, I conducted comprehensive evaluation to ensure the AI system met healthcare-grade standards and could handle real-world deployment challenges.

• Final CNN performance: ~93% accuracy on training data, ~75% accuracy on unseen test data.
• Comprehensive evaluation: Model outputs included confusion matrix analysis and precision/recall/F1 score analysis.
• Bias assessment: Conducted fairness analysis ensuring the model wasn't biased toward one class, maintaining reliability for both benign and malignant classifications.
• Statistical validation: Implemented cross-validation and confidence intervals to ensure robust performance estimates.

• Class imbalance mastery: Successfully beat dataset skewing using weighting techniques and careful preprocessing strategies.
• Database architecture: Solved client-side JavaScript database access limitations by implementing Flask API as middleware.
• Overfitting prevention: Early overfitting issues resolved through regularisation techniques.
• Deployment optimisation: Navigated TensorFlow.js conflicts and ONNX export failures by implementing robust Flask server + .keras model architecture.

• Advanced AI model building: Layer design, hyperparameter tuning, callbacks implementation, comprehensive loss/accuracy tracking.
• Data science expertise: Exploratory data analysis, data cleaning, class balancing, statistical error analysis.
• Full-stack AI deployment: Complete pipeline from Python → MySQL → Flask → JavaScript/HTML/CSS.

Final Word

This project wasn't just a tick-the-box Leaving Cert submission. I trained a deep CNN on medical data, handled all the messy preprocessing and class imbalance challenges, built a proper Flask + MySQL backend, and delivered a live interactive web tool for healthcare professionals.

It proved I can:

• Go deep into AI/ML theory and practice — from basic neural networks to production-ready deep learning systems.
• Build complete pipelines from raw medical data → sophisticated CNN model → full-stack deployment.
• Solve real healthcare problems with code that actually works in clinical environments.
• Handle production challenges including data bias, model deployment, and system integration.

And honestly, being able to say I built an AI that helps spot skin cancer when I'm still in school? That's something I'm genuinely proud of. This project demonstrated my ability to take a complex healthcare challenge and build a comprehensive, production-ready AI solution that could make a real difference.