#  Neuro140/240 Biological and Artificial Intelligence 

 





 Semester:   Spring 

|

 Year offered:  2023 

|

 Link: [Course Website](https://klab.tch.harvard.edu/academia/classes/BAI/bai.html) 

 

 

 

---

## Class Materials:

### Tutorial 4 on Transformers and NLP

In the tutorial I covered

- The paradigm of self-supervised learning
- NLP 101: language representation and statistical modelling of language
- Transformer architecture
- Transformer beyond NLP.

In this notebook [Transformer playground](https://colab.research.google.com/drive/1KIfKcCsu2aFNi0AFJpJsg1CaPrHQ3Uv0?usp=sharing), we covered

- The general structure of transformers.
- Training a toy transformer to speak a highly simplified artificial "English", via probabilistic generative grammar.

 ![encoder-1](/sites/g/files/omnuum11221/files/2025-05/encoder-1.png)

 



 

 

---

 Attachments- [  picture\_as\_pdf  neuro140\_tutorial3\_nlp\_transformer\_ssl.pdf ](/sites/g/files/omnuum11221/files/binxuw/files/neuro140_tutorial3_nlp_transformer_ssl.pdf)
- [  picture\_as\_pdf  neuro140\_tutorial3\_nlp\_transformer\_ssl.pptx ](/sites/g/files/omnuum11221/files/binxuw/files/neuro140_tutorial3_nlp_transformer_ssl.pptx)
 
---