HCIP 2025 Image Captioning Project

This repository contains our HCIP 2025 deep learning project for automatic image caption generation using the Flickr8k dataset.

The project was developed within the context of the Huawei HCIP AI FCAI program and combines computer vision with natural language processing by extracting image features using InceptionV3 and generating captions with an LSTM-based sequence model.

Project Overview

The goal of this project is to generate a natural language description for an input image.

Workflow

1. Download and load the Flickr8k dataset
2. Read and preprocess image captions
3. Clean text data and add sequence tokens
4. Tokenize captions and prepare training sequences
5. Extract image features using pretrained InceptionV3
6. Train an LSTM-based caption generation model
7. Generate captions for unseen images

Technologies Used

• Python
• TensorFlow / Keras
• NumPy
• Pandas
• Matplotlib
• PIL
• NLTK
• Kaggle API

Model Architecture

This project uses a multimodal deep learning architecture:

• Image branch: InceptionV3 extracts visual features from images
• Text branch: an Embedding layer and LSTM process partial captions
• Fusion layer: image and text features are combined
• Output layer: predicts the next word in the caption sequence

Dataset

• Dataset: Flickr8k
• Source: Kaggle
• Each image is associated with multiple human-written captions

File Structure

hcip-2024-image-captioning/
├── HCIP_2024_Image_Captioning.ipynb
├── README.md
├── requirements.txt
└── .gitignore

Notes

• The dataset itself is not uploaded to this repository
• kaggle.json is required locally in order to download the dataset from Kaggle
• Trained model files and large datasets are excluded from GitHub

Academic Context

This project was developed as part of HCIP 2025 and was presented under the Huawei HCIP AI FCAI program context.

Authors

• Amin Tariq Amin Abbas
• Magdi Bakri Mohammed