Backbones
- K2F-B-1Transformer: Attention Is All You NeedAug 2, 2026
Transformer paper note — the sequence model that drops recurrence and convolution entirely and computes representations purely from (self-)attention: scaled dot-product + multi-head attention, positional encodings, encoder-decoder stacks; new WMT'14 SOTA BLEU at a fraction of the training cost, and the backbone every later architecture (BERT, ViT, CroCo, DUSt3R, VGGT) builds on
- K2F-B-2ViT: An Image Is Worth 16×16 WordsAug 2, 2026
Vision Transformer paper note — apply a near-vanilla Transformer encoder directly to a sequence of image patches (16×16), with a learnable [class] token and 1D position embeddings; with enough pre-training data (JFT-300M) it beats ResNet CNNs while using far less pre-training compute, showing large-scale data can substitute for the convolutional inductive bias
- K2F-B-3DPT: Vision Transformers for Dense PredictionAug 2, 2026
DPT (Dense Prediction Transformer) paper note — use a ViT encoder as the backbone for dense per-pixel prediction: reassemble tokens from several transformer stages into image-like feature maps at multiple resolutions, then RefineNet-style fusion upsamples them to a fine-grained output. Global receptive field at every stage → big gains in monocular depth (zero-shot) and semantic segmentation (ADE20K 49.02% mIoU); the dense-prediction head DUSt3R/CroCo later reuse