Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3279 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2398. Min Range: -1.2398

Max Act: 1.2398. Min Act: -0.0000

Data Index: 1374685 (The Pile)

Max Activating Token Index: 426

Text #1

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.8012. Min Act: -0.0000

Data Index: 367803 (The Pile)

Max Activating Token Index: 887

Text #2

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5897. Min Act: -0.0000

Data Index: 1375276 (The Pile)

Max Activating Token Index: 550

Text #3

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5760. Min Act: -0.0000

Data Index: 1246258 (The Pile)

Max Activating Token Index: 664

Text #4

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5497. Min Act: -0.0000

Data Index: 1841137 (The Pile)

Max Activating Token Index: 339

Text #5

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5039. Min Act: -0.0000

Data Index: 116167 (The Pile)

Max Activating Token Index: 70

Text #6

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5188. Min Act: -0.0000

Data Index: 93678 (The Pile)

Max Activating Token Index: 192

Text #7

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4866. Min Act: -0.0000

Data Index: 803098 (The Pile)

Max Activating Token Index: 1014

Text #8

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4919. Min Act: -0.0000

Data Index: 1415328 (The Pile)

Max Activating Token Index: 854

Text #9

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4869. Min Act: -0.0000

Data Index: 1161502 (The Pile)

Max Activating Token Index: 538

Text #10

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5048. Min Act: -0.0000

Data Index: 1338428 (The Pile)

Max Activating Token Index: 232

Text #11

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.5130. Min Act: -0.0000

Data Index: 491646 (The Pile)

Max Activating Token Index: 824

Text #12

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4589. Min Act: -0.0000

Data Index: 1448919 (The Pile)

Max Activating Token Index: 269

Text #13

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4784. Min Act: -0.0000

Data Index: 384671 (The Pile)

Max Activating Token Index: 250

Text #14

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4598. Min Act: -0.0000

Data Index: 1617458 (The Pile)

Max Activating Token Index: 729

Text #15

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4934. Min Act: -0.0000

Data Index: 13135 (The Pile)

Max Activating Token Index: 64

Text #16

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4594. Min Act: -0.0000

Data Index: 1163994 (The Pile)

Max Activating Token Index: 992

Text #17

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.3869. Min Act: -0.0000

Data Index: 1091572 (The Pile)

Max Activating Token Index: 785

Text #18

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4815. Min Act: -0.0000

Data Index: 716236 (The Pile)

Max Activating Token Index: 1020

Text #19

Max Range: 1.2398. Min Range: -1.2398

Max Act: 0.4573. Min Act: -0.0000

Data Index: 617644 (The Pile)

Max Activating Token Index: 558