Cohere Tiny Aya

Tiny Aya Water is a 3.35B-parameter multilingual language model optimized for European and Asia Pacific languages. It uses a 36-layer transformer with 2,048 hidden size, 16 attention heads, and 4 KV heads. The architecture interleaves sliding window attention (4,096 window) with periodic global attention layers for efficient long-range interaction. It supports around 8K context length and uses RoPE for positional encoding. Trained across 70+ languages, it is designed for strong regional performance while remaining efficient for local deployment and downstream adaptation.

Tiny Aya Global is a 3.35B-parameter multilingual language model designed for balanced performance across diverse languages. It uses a 36-layer transformer with 2,048 hidden size, 16 attention heads, and 4 KV heads. The architecture interleaves sliding window attention (4,096 window) with periodic global attention layers to maintain efficiency and full-context interaction. It supports around 8K context length and uses RoPE for positional encoding. Trained on 70+ languages, it is optimized for multilingual understanding, instruction tuning, and deployment in compute-constrained environments.

Tiny Aya Fire is a 3.35B-parameter multilingual language model optimized for South Asian languages. It uses a 36-layer transformer with 2,048 hidden size, 16 attention heads, and 4 KV heads. The architecture interleaves sliding window attention (4,096 window) with periodic global attention layers for efficient long-range interaction. It supports around 8K context length and uses RoPE for positional encoding. Trained across 70+ languages, it is designed for strong regional performance while remaining efficient for local deployment and downstream adaptation.

Tiny Aya Earth is a 3.35B-parameter multilingual language model optimized for West Asian and African languages. It uses a 36-layer transformer with 2,048 hidden size, 16 attention heads, and 4 KV heads. The architecture interleaves sliding window attention (4,096 window) with periodic global attention layers for efficient long-range interaction. It supports around 8K context length and uses RoPE for positional encoding. Trained across 70+ languages, it is designed for strong regional performance while remaining efficient for local deployment and downstream adaptation.

Tiny Aya Base is a 3.35B-parameter multilingual language model designed for efficient deployment and broad language coverage. It uses a 36-layer transformer with 2,048 hidden size, 16 attention heads, and grouped KV heads (4 KV heads). The architecture alternates sliding window attention (4,096 window) with periodic global attention layers for full-sequence interaction. It supports around 8K context in practice and uses RoPE for positional encoding. Trained across 70+ languages, it is optimized for balanced multilingual performance, downstream adaptation, and low-resource environments.