Memory‑Efficient Attention
Attention kernels that reduce memory/time complexity using tiling, FlashAttention, or linearized variants.
Dapp Ahoy! World beyond Web3
•
Archives: Glossary
Archives: Glossary
MCTS (Monte Carlo Tree Search)
Planning algorithm that balances exploration/exploitation by sampling trajectories; used in agents.
Momentum (Optimizer)
Technique that accumulates velocity of gradients to smooth updates; basis for SGD with momentum, Adam.
MLP (Feedforward Layer)
Multi‑layer perceptron used inside transformer blocks for token‑wise transformations.
Matrix Multiplication (MatMul)
Core linear algebra operation dominating transformer compute; optimized with tiling and tensor cores.
Mixed‑Precision Training
Training with lower precision formats to save memory and increase throughput, with loss‑scaling.
Model Quantization
Reducing numerical precision (e.g., FP16→INT8/4) to shrink models and speed inference.
Model Card
Document describing model intended use, data, metrics, risks, and limitations.
Model Alignment
Training and controls to ensure models follow human intent and avoid harmful behavior.
Mixture of Experts (MoE)
Architecture where a router activates subsets of expert MLPs per token, increasing capacity efficiently.
Masked Language Modeling (MLM)
Pretraining objective where tokens are masked and predicted; used in BERT‑style models.
