Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1557 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.6385. Min Act: -0.0000

Data Index: 1968427 (The Pile)

Max Activating Token Index: 701

Text #1

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5734. Min Act: -0.0000

Data Index: 1380233 (The Pile)

Max Activating Token Index: 975

Text #2

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5988. Min Act: -0.0000

Data Index: 1248083 (The Pile)

Max Activating Token Index: 931

Text #3

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.4959. Min Act: -0.0000

Data Index: 783460 (The Pile)

Max Activating Token Index: 188

Text #4

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5938. Min Act: -0.0000

Data Index: 1432377 (The Pile)

Max Activating Token Index: 638

Text #5

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5485. Min Act: -0.0000

Data Index: 371632 (The Pile)

Max Activating Token Index: 97

Text #6

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5223. Min Act: -0.0000

Data Index: 1109407 (The Pile)

Max Activating Token Index: 746

Text #7

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5626. Min Act: -0.0000

Data Index: 1305944 (The Pile)

Max Activating Token Index: 836

Text #8

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5410. Min Act: -0.0000

Data Index: 1599309 (The Pile)

Max Activating Token Index: 443

Text #9

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5574. Min Act: -0.0000

Data Index: 222258 (The Pile)

Max Activating Token Index: 815

Text #10

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5392. Min Act: -0.0000

Data Index: 1015785 (The Pile)

Max Activating Token Index: 246

Text #11

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5525. Min Act: -0.0000

Data Index: 818029 (The Pile)

Max Activating Token Index: 959

Text #12

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.4888. Min Act: -0.0000

Data Index: 321565 (The Pile)

Max Activating Token Index: 34

Text #13

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5172. Min Act: -0.0000

Data Index: 1050020 (The Pile)

Max Activating Token Index: 745

Text #14

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5125. Min Act: -0.0000

Data Index: 703745 (The Pile)

Max Activating Token Index: 243

Text #15

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.4974. Min Act: -0.0000

Data Index: 309632 (The Pile)

Max Activating Token Index: 77

Text #16

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5291. Min Act: -0.0000

Data Index: 559164 (The Pile)

Max Activating Token Index: 570

Text #17

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.5072. Min Act: -0.0000

Data Index: 1437276 (The Pile)

Max Activating Token Index: 348

Text #18

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.4888. Min Act: -0.0000

Data Index: 1531906 (The Pile)

Max Activating Token Index: 149

Text #19

Max Range: 0.6385. Min Range: -0.6385

Max Act: 0.4960. Min Act: -0.0000

Data Index: 1932244 (The Pile)

Max Activating Token Index: 478