This inaugural lecture of Stanford’s CS231N course provides a comprehensive foundation for understanding the intersection of computer vision and deep learning. Professor Fei-Fei Li delivers a masterful overview that traces the evolution of vision from its biological origins 540 million years ago to today’s AI revolution, while Professor Ehsan Adeli outlines the course structure and learning objectives. The lecture serves as both historical context and practical roadmap for one of the most transformative fields in modern artificial intelligence with sample code<\/a>. Video inside<\/a>.<\/p>\n\n\n\n Deep learning for computer vision uses neural networks with multiple layers to automatically learn visual patterns and features from images, rather than relying on hand-crafted features. The key breakthrough is that these networks can learn hierarchical representations – from simple edges and textures in early layers to complex object parts and full objects in deeper layers.<\/p>\n\n\n\n Core Architecture: Convolutional Neural Networks (CNNs)<\/strong><\/p>\n\n\n\n CNNs are specifically designed for visual data and use three main operations:<\/p>\n\n\n\n This demonstrate a real-world application that showcases the power and social impact of deep learning in computer vision with python:<\/p>\n\n\n\n This skin cancer detection example perfectly illustrates the core principles from the Stanford CS231N lecture:<\/p>\n\n\n\n Hierarchical Feature Learning<\/strong> The CNN automatically learns a hierarchy of visual features, just like the biological visual system described by Hubel and Wiesel:<\/p>\n\n\n\n End-to-End Learning<\/strong> Unlike traditional approaches that required hand-crafted features, this CNN learns everything from raw pixels to diagnosis automatically, demonstrating the revolutionary shift that occurred with the 2012 ImageNet breakthrough.<\/p>\n\n\n\n This application showcases computer vision’s potential for social good:<\/p>\n\n\n\n Medical Accessibility<\/strong><\/p>\n\n\n\n Clinical Performance<\/strong><\/p>\n\n\n\n Technical Robustness<\/strong><\/p>\n\n\n\n This application demonstrates several key advances in deep learning for computer vision:<\/p>\n\n\n\n This exemplifies how computer vision has evolved from the simple edge detection experiments of the 1980s to sophisticated systems that can assist in life-saving medical diagnoses, perfectly illustrating the transformative journey described in Professor Li’s lecture.<\/p>\n\n\n\nDeep Learning for Computer Vision<\/h2>\n\n\n\n
\n
Practical Example: Medical Image Analysis for Skin Cancer Detection<\/h3>\n\n\n\n
# ============================================================================\n# COMPLETE SETUP AND EXECUTION GUIDE FOR SKIN CANCER DETECTION\n# ============================================================================\n\n\"\"\"\nSTEP 1: ENVIRONMENT SETUP\n========================\n\nFirst, create a virtual environment and install dependencies:\n\nconda create -n skin_cancer python=3.9\nconda activate skin_cancer\n\npip install torch torchvision torchaudio --index-url https:\/\/download.pytorch.org\/whl\/cu118\npip install pillow matplotlib numpy pandas scikit-learn kaggle\n\n# For CPU-only version (if no GPU):\n# pip install torch torchvision torchaudio --index-url https:\/\/download.pytorch.org\/whl\/cpu\n\"\"\"\n\nimport os\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport torchvision.transforms as transforms\nfrom torch.utils.data import DataLoader, Dataset\nfrom PIL import Image\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport pandas as pd\nfrom sklearn.model_selection import train_test_split\nimport zipfile\nimport kaggle\n\n# ============================================================================\n# STEP 2: DATA DOWNLOAD AND PREPARATION\n# ============================================================================\n\ndef setup_kaggle_and_download_data():\n \"\"\"\n Download the HAM10000 skin lesion dataset from Kaggle\n \"\"\"\n print(\"Setting up Kaggle API and downloading data...\")\n \n # Note: You need to set up Kaggle API credentials first\n # 1. Go to kaggle.com -> Account -> API -> Create New API Token\n # 2. Place kaggle.json in ~\/.kaggle\/ (Linux\/Mac) or C:\\Users\\{username}\\.kaggle\\ (Windows)\n \n # Download the dataset\n kaggle.api.dataset_download_files(\n 'kmader\/skin-cancer-mnist-ham10000', \n path='.\/data', \n unzip=True\n )\n print(\"Dataset downloaded successfully!\")\n\ndef prepare_dataset():\n \"\"\"\n Prepare the dataset for training\n \"\"\"\n # Load metadata\n metadata_path = '.\/data\/HAM10000_metadata.csv'\n if not os.path.exists(metadata_path):\n print(\"Please download the HAM10000 dataset first using setup_kaggle_and_download_data()\")\n return None\n \n df = pd.read_csv(metadata_path)\n \n # Create class mapping\n class_mapping = {\n 'akiec': 0, # Actinic keratoses\n 'bcc': 1, # Basal cell carcinoma\n 'bkl': 2, # Benign keratosis\n 'df': 3, # Dermatofibroma\n 'mel': 4, # Melanoma\n 'nv': 5, # Melanocytic nevi\n 'vasc': 6 # Vascular lesions\n }\n \n df['label'] = df['dx'].map(class_mapping)\n \n # Create image paths\n def get_image_path(image_id):\n for folder in ['HAM10000_images_part_1', 'HAM10000_images_part_2']:\n path = f'.\/data\/{folder}\/{image_id}.jpg'\n if os.path.exists(path):\n return path\n return None\n \n df['image_path'] = df['image_id'].apply(get_image_path)\n df = df.dropna(subset=['image_path']) # Remove missing images\n \n # Split dataset\n train_df, test_df = train_test_split(df, test_size=0.2, stratify=df['label'], random_state=42)\n train_df, val_df = train_test_split(train_df, test_size=0.2, stratify=train_df['label'], random_state=42)\n \n print(f\"Dataset split: Train={len(train_df)}, Val={len(val_df)}, Test={len(test_df)}\")\n return train_df, val_df, test_df, class_mapping\n\n# ============================================================================\n# STEP 3: DATASET CLASS IMPLEMENTATION\n# ============================================================================\n\nclass SkinLesionDataset(Dataset):\n def __init__(self, dataframe, transform=None):\n self.df = dataframe.reset_index(drop=True)\n self.transform = transform\n \n def __len__(self):\n return len(self.df)\n \n def __getitem__(self, idx):\n img_path = self.df.iloc[idx]['image_path']\n label = self.df.iloc[idx]['label']\n \n try:\n image = Image.open(img_path).convert('RGB')\n \n if self.transform:\n image = self.transform(image)\n \n return image, label\n except Exception as e:\n print(f\"Error loading image {img_path}: {e}\")\n # Return a black image and label 0 as fallback\n if self.transform:\n image = self.transform(Image.new('RGB', (224, 224), (0, 0, 0)))\n else:\n image = Image.new('RGB', (224, 224), (0, 0, 0))\n return image, 0\n\n# ============================================================================\n# STEP 4: MODEL DEFINITION (Same as before)\n# ============================================================================\n\nclass SkinCancerCNN(nn.Module):\n def __init__(self, num_classes=7):\n super(SkinCancerCNN, self).__init__()\n \n self.features = nn.Sequential(\n nn.Conv2d(3, 32, kernel_size=3, padding=1),\n nn.BatchNorm2d(32),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=2, stride=2),\n \n nn.Conv2d(32, 64, kernel_size=3, padding=1),\n nn.BatchNorm2d(64),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=2, stride=2),\n \n nn.Conv2d(64, 128, kernel_size=3, padding=1),\n nn.BatchNorm2d(128),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=2, stride=2),\n \n nn.Conv2d(128, 256, kernel_size=3, padding=1),\n nn.BatchNorm2d(256),\n nn.ReLU(inplace=True),\n nn.MaxPool2d(kernel_size=2, stride=2),\n \n nn.AdaptiveAvgPool2d((1, 1))\n )\n \n self.classifier = nn.Sequential(\n nn.Dropout(0.5),\n nn.Linear(256, 128),\n nn.ReLU(inplace=True),\n nn.Dropout(0.3),\n nn.Linear(128, num_classes)\n )\n \n def forward(self, x):\n x = self.features(x)\n x = x.view(x.size(0), -1)\n x = self.classifier(x)\n return x\n\n# ============================================================================\n# STEP 5: TRAINING FUNCTION\n# ============================================================================\n\ndef train_model_complete(train_df, val_df, num_epochs=20):\n \"\"\"\n Complete training function with data loading\n \"\"\"\n # Set up transforms\n train_transforms = transforms.Compose([\n transforms.Resize((224, 224)),\n transforms.RandomHorizontalFlip(p=0.5),\n transforms.RandomRotation(degrees=15),\n transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2),\n transforms.ToTensor(),\n transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n \n val_transforms = transforms.Compose([\n transforms.Resize((224, 224)),\n transforms.ToTensor(),\n transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n \n # Create datasets\n train_dataset = SkinLesionDataset(train_df, transform=train_transforms)\n val_dataset = SkinLesionDataset(val_df, transform=val_transforms)\n \n # Create data loaders\n train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True, num_workers=4)\n val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False, num_workers=4)\n \n # Initialize model\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n model = SkinCancerCNN(num_classes=7)\n model = model.to(device)\n \n print(f\"Training on device: {device}\")\n print(f\"Model has {sum(p.numel() for p in model.parameters())} parameters\")\n \n # Loss and optimizer\n criterion = nn.CrossEntropyLoss()\n optimizer = optim.Adam(model.parameters(), lr=0.001, weight_decay=1e-4)\n scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', patience=5)\n \n # Training loop\n best_val_acc = 0.0\n train_losses, val_accuracies = [], []\n \n for epoch in range(num_epochs):\n # Training\n model.train()\n running_loss = 0.0\n \n for batch_idx, (images, labels) in enumerate(train_loader):\n images, labels = images.to(device), labels.to(device)\n \n optimizer.zero_grad()\n outputs = model(images)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.step()\n \n running_loss += loss.item()\n \n if batch_idx % 50 == 0:\n print(f'Epoch {epoch+1}\/{num_epochs}, Batch {batch_idx}\/{len(train_loader)}, Loss: {loss.item():.4f}')\n \n # Validation\n model.eval()\n correct = 0\n total = 0\n val_loss = 0.0\n \n with torch.no_grad():\n for images, labels in val_loader:\n images, labels = images.to(device), labels.to(device)\n outputs = model(images)\n loss = criterion(outputs, labels)\n val_loss += loss.item()\n \n _, predicted = torch.max(outputs.data, 1)\n total += labels.size(0)\n correct += (predicted == labels).sum().item()\n \n val_accuracy = 100 * correct \/ total\n avg_train_loss = running_loss \/ len(train_loader)\n avg_val_loss = val_loss \/ len(val_loader)\n \n train_losses.append(avg_train_loss)\n val_accuracies.append(val_accuracy)\n \n scheduler.step(avg_val_loss)\n \n if val_accuracy > best_val_acc:\n best_val_acc = val_accuracy\n torch.save(model.state_dict(), 'best_skin_cancer_model.pth')\n print(f'New best model saved with validation accuracy: {val_accuracy:.2f}%')\n \n print(f'Epoch [{epoch+1}\/{num_epochs}], Train Loss: {avg_train_loss:.4f}, Val Accuracy: {val_accuracy:.2f}%')\n print('-' * 50)\n \n return model, train_losses, val_accuracies\n\n# ============================================================================\n# STEP 6: PREDICTION FUNCTION\n# ============================================================================\n\ndef predict_single_image(model_path, image_path, class_names):\n \"\"\"\n Predict on a single image\n \"\"\"\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n \n # Load model\n model = SkinCancerCNN(num_classes=7)\n model.load_state_dict(torch.load(model_path, map_location=device))\n model = model.to(device)\n model.eval()\n \n # Preprocess image\n val_transforms = transforms.Compose([\n transforms.Resize((224, 224)),\n transforms.ToTensor(),\n transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n ])\n \n image = Image.open(image_path).convert('RGB')\n image_tensor = val_transforms(image).unsqueeze(0).to(device)\n \n with torch.no_grad():\n outputs = model(image_tensor)\n probabilities = torch.softmax(outputs, dim=1)\n confidence, predicted = torch.max(probabilities, 1)\n \n # Get top 3 predictions\n top3_prob, top3_indices = torch.topk(probabilities, 3)\n \n print(f\"Predicted class: {class_names[predicted.item()]}\")\n print(f\"Confidence: {confidence.item():.3f}\")\n print(\"\\nTop 3 predictions:\")\n for i, (prob, idx) in enumerate(zip(top3_prob[0], top3_indices[0])):\n print(f\"{i+1}. {class_names[idx]}: {prob:.3f}\")\n \n return predicted.item(), confidence.item()\n\n# ============================================================================\n# STEP 7: MAIN EXECUTION FUNCTION\n# ============================================================================\n\ndef main():\n \"\"\"\n Main execution function - runs the complete pipeline\n \"\"\"\n print(\"=\" * 60)\n print(\"SKIN CANCER DETECTION WITH DEEP LEARNING\")\n print(\"=\" * 60)\n \n # Class names\n class_names = [\n 'Actinic keratoses',\n 'Basal cell carcinoma', \n 'Benign keratosis',\n 'Dermatofibroma',\n 'Melanoma',\n 'Melanocytic nevi',\n 'Vascular lesions'\n ]\n \n print(\"Step 1: Checking for dataset...\")\n if not os.path.exists('.\/data\/HAM10000_metadata.csv'):\n print(\"Dataset not found. Please run setup_kaggle_and_download_data() first\")\n print(\"Make sure you have Kaggle API set up with credentials\")\n return\n \n print(\"Step 2: Preparing dataset...\")\n train_df, val_df, test_df, class_mapping = prepare_dataset()\n \n print(\"Step 3: Starting training...\")\n model, train_losses, val_accuracies = train_model_complete(train_df, val_df, num_epochs=10)\n \n print(\"Step 4: Training completed!\")\n print(f\"Best validation accuracy: {max(val_accuracies):.2f}%\")\n \n # Plot training curves\n plt.figure(figsize=(12, 4))\n \n plt.subplot(1, 2, 1)\n plt.plot(train_losses)\n plt.title('Training Loss')\n plt.xlabel('Epoch')\n plt.ylabel('Loss')\n \n plt.subplot(1, 2, 2)\n plt.plot(val_accuracies)\n plt.title('Validation Accuracy')\n plt.xlabel('Epoch')\n plt.ylabel('Accuracy (%)')\n \n plt.tight_layout()\n plt.savefig('training_curves.png')\n plt.show()\n \n print(\"Step 5: Testing prediction on a sample image...\")\n # Test on a random image from test set\n test_image_path = test_df.iloc[0]['image_path']\n predict_single_image('best_skin_cancer_model.pth', test_image_path, class_names)\n\n# ============================================================================\n# STEP 8: QUICK START FOR TESTING (WITHOUT FULL TRAINING)\n# ============================================================================\n\ndef quick_demo():\n \"\"\"\n Quick demo with a pre-trained model (you would need to download or train first)\n \"\"\"\n print(\"Quick Demo Mode\")\n print(\"Note: This requires a pre-trained model file\")\n \n # Create a dummy model for demonstration\n model = SkinCancerCNN(num_classes=7)\n torch.save(model.state_dict(), 'demo_model.pth')\n \n class_names = [\n 'Actinic keratoses', 'Basal cell carcinoma', 'Benign keratosis',\n 'Dermatofibroma', 'Melanoma', 'Melanocytic nevi', 'Vascular lesions'\n ]\n \n print(\"Demo model created. In practice, you would:\")\n print(\"1. Train the model with real data\")\n print(\"2. Save the trained weights\")\n print(\"3. Load for inference\")\n\nif __name__ == \"__main__\":\n # Choose execution mode\n print(\"Choose execution mode:\")\n print(\"1. Full pipeline (requires Kaggle setup and HAM10000 dataset)\")\n print(\"2. Quick demo (creates dummy model)\")\n \n choice = input(\"Enter choice (1 or 2): \")\n \n if choice == \"1\":\n main()\n else:\n quick_demo()\n\n# ============================================================================\n# ADDITIONAL SETUP INSTRUCTIONS\n# ============================================================================\n\n\"\"\"\nCOMPLETE SETUP INSTRUCTIONS:\n============================\n\n1. ENVIRONMENT SETUP:\n conda create -n skin_cancer python=3.9\n conda activate skin_cancer\n pip install torch torchvision matplotlib pandas scikit-learn kaggle pillow\n\n2. KAGGLE API SETUP:\n - Go to kaggle.com\n - Account -> API -> Create New API Token\n - Download kaggle.json\n - Place in ~\/.kaggle\/ (Linux\/Mac) or C:\\Users\\{username}\\.kaggle\\ (Windows)\n - chmod 600 ~\/.kaggle\/kaggle.json (Linux\/Mac)\n\n3. DOWNLOAD DATA:\n python -c \"from skin_cancer_detection import setup_kaggle_and_download_data; setup_kaggle_and_download_data()\"\n\n4. RUN TRAINING:\n python skin_cancer_detection.py\n\n5. USE TRAINED MODEL:\n python -c \"from skin_cancer_detection import predict_single_image; predict_single_image('best_skin_cancer_model.pth', 'path_to_image.jpg', class_names)\"\n\nHARDWARE REQUIREMENTS:\n- GPU recommended (NVIDIA with CUDA support)\n- Minimum 8GB RAM\n- ~2GB storage for dataset\n- Training time: 2-4 hours on GPU, 8-12 hours on CPU\n\nTROUBLESHOOTING:\n- CUDA out of memory: Reduce batch_size in DataLoader\n- Dataset download fails: Check Kaggle API credentials\n- Training too slow: Use GPU or reduce num_epochs\n- Import errors: Check all dependencies are installed\n\"\"\"<\/code><\/pre>\n\n\n\n\n
Real-World Impact<\/h3>\n\n\n\n
\n
\n
\n
Why This Example Matters<\/h3>\n\n\n\n
\n
Video of understanding the intersection of CV<\/h2>\n\n\n\n