Hugging Face & Modern NLP Masterclass Roadmap

A comprehensive, structured learning repository covering fundamental to advanced Natural Language Processing (NLP), Classical Vectorization, Deep Learning Embeddings, Hugging Face Ecosystem, and complete Transformer Architectures.

---

📚 Complete Learning Notebook Sequence

#	Notebook	Focus Area	Key Concepts Covered
01	01_intro.ipynb	NLP & Hugging Face Intro	Pipelines for sentiment analysis, text generation, NER, and QA.
02	02_login.ipynb	Hub Authentication	huggingface_hub authentication, .env token management, security best practices.
03	03_dataset.ipynb	Datasets Library	Loading, filtering, mapping, splitting, and inspecting Hugging Face datasets.
04	04_custom_dataset.ipynb	Custom Data Pipelines	Building custom HF datasets from CSV, JSON, and Pandas DataFrames.
05	05_tokenization.ipynb	Tokenization Foundations	AutoTokenizer, Subword (BPE, WordPiece), token IDs, padding, and attention masks.
06	06_train_own_tokenizer.ipynb	Custom Tokenizer Training	Training a custom WordPiece/BPE tokenizer from scratch and saving artifacts.
07	07_stemming_lemmatization.ipynb	Classical Text Normalization	Stemming (Porter, Snowball) vs. Lemmatization (WordNet, NLTK, spaCy).
08	08_ner_and_pos.ipynb	Information Extraction	Named Entity Recognition (NER) & Part-of-Speech (POS) tagging pipelines.
09	09_classical_encoding_tfidf.ipynb	Classical Vectorization	One-Hot Encoding, Bag-of-Words (BoW), TF-IDF matrix feature extraction.
10	10_word2vec_skipgram.ipynb	Static Word Embeddings	Word2Vec architecture (CBOW vs. Skip-Gram), Gensim embeddings, cosine similarity.
11	11_embeddings.ipynb	Contextual Embeddings	Hugging Face AutoModel feature extraction, mean/CLS pooling, and task-specific heads.
12	12_download_model.ipynb	Local Model Management	Offline model caching, snapshot downloading, and local inference execution.
13	13_evaluation_metrics.ipynb	Model Evaluation	Hugging Face evaluate, Accuracy, Precision, Recall, F1, BLEU, ROUGE, Perplexity.
14	14_hf_api.ipynb	Hugging Face Hub APIs	InferenceClient, Serverless API endpoints, and remote model repository management.
15	15_Transformer_Architecture.ipynb	Transformer Deep-Dive	Complete Vaswani et al. architecture, Encoder-Decoder stacks, Multi-Head Self-Attention ($Q, K, V$), Scaling factor $\sqrt{d_k}$ proof, Sinusoidal Positional Encodings, Feed-Forward Networks, LayerNorm & Residuals, Causal Masking, Cross-Attention, Teacher Forcing vs. Autoregressive inference, pure PyTorch implementation.

---

🏗️ Transformer Architecture Pipeline Overview

Below is the high-level workflow of the Encoder-Decoder Transformer architecture detailed in 15_Transformer_Architecture.ipynb:

Key Architectural Components

1. Multi-Head Self-Attention (MHSA): Computes dynamic contextual token interactions using Query ($Q$), Key ($K$), and Value ($V$) projections: $$\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V$$
2. Positional Encoding: Injects sequence order using sinusoidal functions across embedding dimensions: $$\mathrm{PE}{(pos, 2i)} = \sin\left(\frac{pos}{10000^{2i/d}}\right), \quad \mathrm{PE}{(pos, 2i+1)} = \cos\left(\frac{pos}{10000^{2i/d}}\right)$$
3. Position-wise Feed-Forward Network (FFN): Processes representations independently per token ($d_{model} \to 4d_{model} \to d_{model}$).
4. Causal Masked Decoder Attention: Prevents future token visibility during autoregressive generation.

---

🛠️ Environment Setup & Installation

1. Prerequisites

• Python 3.9+ installed
• Virtual environment tool (venv or conda)

2. Step-by-Step Installation

# Clone the repository
git clone https://github.com/Saimtec/NLP-Learning.git

# Create and activate virtual environment
python -m venv .venv

# On Windows (PowerShell):
.venv\Scripts\Activate.ps1

# On Linux/macOS:
source .venv/bin/activate

# Install required dependencies
pip install -r requirements.txt

3. Hugging Face Authentication Setup

Create a .env file in the root directory:

HF_TOKEN=your_huggingface_access_token_here

Security Note: Never commit your .env file or hardcode tokens into Jupyter Notebook cells. The .gitignore file is configured to exclude sensitive files.

---

📂 Project Directory Structure

nlp-ml-learning/
│
├── notebooks/                              # Sequenced Learning Notebooks
│   ├── 01_intro.ipynb                      # Intro & HF Pipelines
│   ├── 02_login.ipynb                      # HF Hub Authentication
│   ├── 03_dataset.ipynb                    # Datasets Library
│   ├── 04_custom_dataset.ipynb             # Custom Dataset Construction
│   ├── 05_tokenization.ipynb               # Tokenization Mechanics
│   ├── 06_train_own_tokenizer.ipynb        # Tokenizer Training
│   ├── 07_stemming_lemmatization.ipynb     # Classical Normalization
│   ├── 08_ner_and_pos.ipynb                # Information Extraction
│   ├── 09_classical_encoding_tfidf.ipynb   # TF-IDF & BoW
│   ├── 10_word2vec_skipgram.ipynb          # Word2Vec Embeddings
│   ├── 11_embeddings.ipynb                 # Contextual Embeddings
│   ├── 12_download_model.ipynb             # Local Model Management
│   ├── 13_evaluation_metrics.ipynb         # Evaluation & Metrics
│   ├── 14_hf_api.ipynb                     # Serverless & Hub API
│   ├── 15_Transformer_Architecture.ipynb  # Transformer Masterclass Notes
│   └── images/                             # High-Resolution Architectural Diagrams
│       ├── 01_transformer_full_pipeline.png
│       ├── 02_encoder_block.png
│       ├── 03_decoder_block.png
│       └── 04_qkv_attention_pipeline.png
│
├── artifacts/                              # Generated Outputs (Tokenizer/Model artifacts)
├── .env.example                            # Template for environment configuration
├── .gitignore                              # Git exclusion rules
├── requirements.txt                        # Project Python dependencies
└── README.md                               # Repository Documentation

---

📄 License & Acknowledgments

This repository is maintained for practical NLP learning and deep-dive research into modern language models.

• Reference Paper: "Attention Is All You Need" (Vaswani et al., 2017).