Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4440 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1150. Min Range: -1.1150

Max Act: 1.1150. Min Act: -0.0000

Data Index: 878033 (The Pile)

Max Activating Token Index: 55

Text #1

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9700. Min Act: -0.0000

Data Index: 223642 (The Pile)

Max Activating Token Index: 66

Text #2

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9267. Min Act: -0.0000

Data Index: 1229286 (The Pile)

Max Activating Token Index: 469

Text #3

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9008. Min Act: -0.0000

Data Index: 1424674 (The Pile)

Max Activating Token Index: 357

Text #4

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.8729. Min Act: -0.0000

Data Index: 864256 (The Pile)

Max Activating Token Index: 694

Text #5

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9391. Min Act: -0.0000

Data Index: 1028988 (The Pile)

Max Activating Token Index: 40

Text #6

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.8681. Min Act: -0.0000

Data Index: 1390343 (The Pile)

Max Activating Token Index: 497

Text #7

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9501. Min Act: -0.0000

Data Index: 190796 (The Pile)

Max Activating Token Index: 481

Text #8

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.8600. Min Act: -0.0000

Data Index: 1069593 (The Pile)

Max Activating Token Index: 461

Text #9

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9117. Min Act: -0.0000

Data Index: 1811895 (The Pile)

Max Activating Token Index: 571

Text #10

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.9301. Min Act: -0.0000

Data Index: 1028850 (The Pile)

Max Activating Token Index: 860

Text #11

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7330. Min Act: -0.0000

Data Index: 1341936 (The Pile)

Max Activating Token Index: 661

Text #12

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7792. Min Act: -0.0000

Data Index: 317792 (The Pile)

Max Activating Token Index: 52

Text #13

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.8108. Min Act: -0.0000

Data Index: 1372171 (The Pile)

Max Activating Token Index: 484

Text #14

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7239. Min Act: -0.0000

Data Index: 1872296 (The Pile)

Max Activating Token Index: 97

Text #15

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7356. Min Act: -0.0000

Data Index: 1612044 (The Pile)

Max Activating Token Index: 538

Text #16

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7169. Min Act: -0.0000

Data Index: 1389073 (The Pile)

Max Activating Token Index: 444

Text #17

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.7613. Min Act: -0.0000

Data Index: 1660325 (The Pile)

Max Activating Token Index: 72

Text #18

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.6336. Min Act: -0.0000

Data Index: 639317 (The Pile)

Max Activating Token Index: 552

Text #19

Max Range: 1.1150. Min Range: -1.1150

Max Act: 0.6922. Min Act: -0.0000

Data Index: 388621 (The Pile)

Max Activating Token Index: 194