hidden_size % num_attention_heads != 0

by mseeger - opened Dec 12, 2024

Dec 12, 2024

This would have head_size=85, but then 85*12 = 1020 != 1024. Does this even work?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment