Engram

This repository contains the official implementation for Engram, which explores conditional memory as a complementary sparsity axis to MoE. It modernizes classic N-gram embeddings for O(1) lookup to work directly alongside the Transformer backbone.