teaching

Welcome to my teaching materials collection. ⚠️ These page and course are currently under development.

Transformer Architectures

End-to-end implementation of transformer architectures with production-grade practices.

Part 1: Core Implementation

GitHub
Notebook English Russian
1. Understanding Tokenization: Byte-Pair Encoding Kaggle Colab Kaggle Colab
2. Implementing Transformer Architecture Kaggle Colab Kaggle Colab
3. Improving Architecture with SoTA Techniques Kaggle Colab Kaggle Colab
4. Evaluation Metrics: BLEU, ROUGE, METEOR, WandB Kaggle Colab Kaggle Colab
5. Hyperparameter tuning with Wandb Sweeps Kaggle Colab Kaggle Colab
6. Complete Transformer: End-to-End Python Pipeline Kaggle Colab Kaggle Colab
7. Create API and use Gradio and Streamlit Kaggle Colab Kaggle Colab
8. Model Deployment and Monitoring Kaggle Colab Kaggle Colab

Part 2: Transformer-Based Architectures

GitHub
Notebook English Russian
1. BERT: Bidirectional Encoder Representations GitHub Kaggle
2. GPT: Generative Pre-trained Transformers GitHub Kaggle
3. T5: Text-to-Text Transfer Framework GitHub Kaggle
4. Mixture of Experts: Switch Transformers GitHub Kaggle
5. Vision Transformers: ViT, DeiT GitHub Kaggle
6. Longformers: Efficient Long-Context Attention GitHub
7. Multimodal Architectures: CLIP, Llava GitHub
8. LLM Architectures: Gemma2, LLaMA, Mistral GitHub