1 contributor

History: 7 commits

SQCU

compiled models train faster so you can train more of them in a short experiment, to better convergence.

921107d verified 20 days ago

basic_pgptlformer-e118dbaf-93da-4b79-8ab7-ed696e2d80bc
Upload 8 files 27 days ago
dyn_qkl2norm-f68a7719-e9c8-4329-8177-19a229adf90c
Upload 8 files 27 days ago
dyn_qkrmsnorm-3824bd2b-dd4b-4737-b074-57574f2cd8fc
Upload 8 files 27 days ago
dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
qkrmsnorm-a283dcc5-027f-4618-8e8f-460bef6b2739
Upload 8 files 27 days ago
re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e
sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-ATTNII_rev2-d46b31ce-94f1-41c7-89cb-358a7a3f316b
compiled models train faster so you can train more of them in a short experiment, to better convergence. 20 days ago
re-pqt-rmsXrmsx2x2-ATTNII-791967c5-5c59-4a5f-a2c5-07772bcf65ab
89,301,000 parameter attention_ii, z_lossed model trained for 6250 steps at batchsize:4*32, device_batchsize:32 22 days ago
re-pqt-rmsXrmsx3-ATTNII_fast-af1c5037-28b9-4ce3-b351-c78046f90ee7
compiled models train faster so you can train more of them in a short experiment, to better convergence. 20 days ago
.gitattributes

1.52 kB

initial commit 27 days ago
README.md

712 Bytes

Update README.md 27 days ago
dyn_qkl2norm-f68a7719-e9c8-4329-8177-19a229adf90c.txt

474 kB

Upload 8 files 27 days ago
dyn_qkrmsnorm-3824bd2b-dd4b-4737-b074-57574f2cd8fc.txt

474 kB

Upload 8 files 27 days ago
dyn_qkrmsnorm_ii-7a038ecd-be98-46cb-abe8-e0f013fd7eed.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
e118dbaf-93da-4b79-8ab7-ed696e2d80bc.txt

474 kB

Upload 8 files 27 days ago
qkrmsnorm-a283dcc5-027f-4618-8e8f-460bef6b2739.txt

473 kB

Upload 8 files 27 days ago
re-pqt-rmsXrms-2fd4a7cb-930a-464b-90e0-eef9d7551d8d.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-2x-42e14b65-2277-45ae-a68c-822eb66be09a.txt

474 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-ATTNII-697f0113-bb05-480b-b6dc-42a97de0de3e.txt

576 kB

sling the illustrious and mysterious "attention_II" models. also some layerwise rmsnorm, qkprojection rmsnorm models, one twice as large as the other. 22 days ago
re-pqt-rmsXrms-ATTNII_rev2-d46b31ce-94f1-41c7-89cb-358a7a3f316b.txt

570 kB

compiled models train faster so you can train more of them in a short experiment, to better convergence. 20 days ago
re-pqt-rmsXrmsx2x2-ATTNII-791967c5-5c59-4a5f-a2c5-07772bcf65ab.txt

585 kB

89,301,000 parameter attention_ii, z_lossed model trained for 6250 steps at batchsize:4*32, device_batchsize:32 22 days ago
re-pqt-rmsXrmsx3-ATTNII_fast-af1c5037-28b9-4ce3-b351-c78046f90ee7.txt

3.72 MB

compiled models train faster so you can train more of them in a short experiment, to better convergence. 20 days ago