Architectures
- K2F-B-1Transformer: Attention Is All You NeedAug 2, 2026
Transformer paper note — the sequence model that drops recurrence and convolution entirely and computes representations purely from (self-)attention: scaled dot-product + multi-head attention, positional encodings, encoder-decoder stacks; new WMT'14 SOTA BLEU at a fraction of the training cost, and the backbone every later architecture (BERT, ViT, CroCo, DUSt3R, VGGT) builds on
- K2F-B-2ViT: An Image Is Worth 16×16 WordsAug 2, 2026
Vision Transformer paper note — apply a near-vanilla Transformer encoder directly to a sequence of image patches (16×16), with a learnable [class] token and 1D position embeddings; with enough pre-training data (JFT-300M) it beats ResNet CNNs while using far less pre-training compute, showing large-scale data can substitute for the convolutional inductive bias