SQCU
/

pgptlformer-tinystories

SQCU commited on 22 days ago

Commit

1f45909

verified ·

1 Parent(s): 6d543db

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other.

Files changed (8) hide show

dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed/state_step006250.pt ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:e311884b9669c3546ffa583b7f3c61e275ca46a5c33404356c69222f2f124404
+size 334566132

re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d/state_step006250.pt ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:f156ded20576bc820aeb11d0e43bb02f8ace74961b505e1f48d23b81b8205b14
+size 334566068

re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a/state_step006250.pt ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:7b2d57c1b45e1c1f04d9cb6d781589e3778c5fb6ddaf4a75762492825d20d1da
+size 719554682

re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e/state_step006250.pt ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:2d88e2f0adb9256f683d1649484454420b85f69ab7fb7600a5b251495cb5e573
+size 337741964