RC Gate or Sigmoid Gate — Which Would You Choose to Train a Transformer?
Published:
A story about a beautiful idea that didn’t work, and what I learned from killing it.
Posts by Tags
machine learning
research
RC Gate or Sigmoid Gate — Which Would You Choose to Train a Transformer?
Published:
A story about a beautiful idea that didn’t work, and what I learned from killing it.
transformers
RC Gate or Sigmoid Gate — Which Would You Choose to Train a Transformer?
Published:
A story about a beautiful idea that didn’t work, and what I learned from killing it.