project 08
Hybrid ViT-BiGRU Pipeline
A hybrid deep learning pipeline capturing spatial features with a Vision Transformer and temporal patterns with bidirectional GRU networks.
- Category
- AI
- Role
- Deep learning researcher and model developer
- Year
- 2026
technologies used
- Python
- PyTorch
- TensorFlow
- NumPy
- Pandas
- Seaborn
- CUDA
problem
Problem
Purely convolutional or purely recurrent architectures cannot capture two things at once in sequential visual data: the spatial detail within a frame and the long-range temporal context across frames. Strengthening one weakens the other.
solution
Solution
I coded a hybrid pipeline in which Vision Transformer blocks act as the spatial feature extractor and their output vectors feed into bidirectional BiGRU layers. I set up CUDA-accelerated training and validation loops on Python, PyTorch and TensorFlow, and visualised the results through confusion matrices, ROC curves and training metrics.
outcome
Outcome
The outcome is a model architecture suited to both academic and practical use, delivering a measurable increase in accuracy and F1-score on complex temporal datasets compared to standard CNN-LSTM models.
key features
- 01ViT blocks as the spatial feature extractor, feeding their vectors into BiGRU layers.
- 02CUDA-accelerated training and validation loops on PyTorch and TensorFlow.
- 03Data preprocessing, augmentation and regularisation techniques to prevent overfitting.
- 04Visualisation of confusion matrices, ROC curves and training metrics.
why these choices
Why BiGRU instead of LSTM?
GRU runs on fewer parameters than LSTM, which significantly reduced training time and memory consumption. Because the structure is bidirectional, the model still sees both past and future context — so what was saved in parameters wasn't lost in performance.
Why ViT instead of a CNN?
Convolutional layers capture local patterns well but struggle to relate distant regions of a frame. The self-attention structure of a Vision Transformer evaluates relationships across the whole frame in one step, feeding a richer feature vector into the sequence model.