#  Tutorial on Transformer model came out in ML from Scratch seminar series 

 





April 19, 2023

 

 

   ![Attention](/sites/g/files/omnuum11221/files/styles/hwp_1_1__360x360_scale/public/AttentionSchematics_white-01.png?itok=ahqF1JlI) 

 

Official [Github repo](https://github.com/Animadversio/TransformerFromScratch) in ML from Scratch seminar series.

- Lecture Slides ([slides ](/file_url/168)):
    - NLP101
    - Transformer architecture
    - Training objective of language model and usage after training
    - Transformer beyond language
- Exercise notebooks for the seminar 
    - Building your self-attention mechanism and transformer architecture (GPT2) from scratch ([Open in Colab](https://colab.research.google.com/drive/1ZuhA6khlWm57WGZ8i38JH-gc5aJrvpvs?usp=sharing))
    - Pretrained Transformers (BERT, GPT) and training transformer on simplified language ([Open in Colab](https://colab.research.google.com/drive/1zZYzAopL__LW4glruSF9lnZYlEmSVI8j?usp=sharing))
    - Notes on Einstein summation rule. ([Open in Colab](https://colab.research.google.com/drive/1mizzN7iRlS2Du5TJvv7Wz7ecKOnpHzrQ?usp=sharing))



 

 

 



 

 

 Share on:- [     Facebook ](#)
- [     Twitter ](#)
- [     Linkedin ](#)