Here we use Alternative Architectures to refer to all model architectures that are not Transformers.