Blog

RC Gate or Sigmoid Gate — Which Would You Choose to Train a Transformer?

10 minute read

Published: February 13, 2025

A story about a beautiful idea that didn’t work, and what I learned from killing it.