Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4236 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.1653. Min Act: -0.0000

Data Index: 814302 (The Pile)

Max Activating Token Index: 113

Text #1

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8767. Min Act: -0.0000

Data Index: 244845 (The Pile)

Max Activating Token Index: 384

Text #2

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8954. Min Act: -0.0000

Data Index: 1853907 (The Pile)

Max Activating Token Index: 656

Text #3

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.1215. Min Act: -0.0000

Data Index: 631293 (The Pile)

Max Activating Token Index: 453

Text #4

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.1830. Min Act: -0.0000

Data Index: 1432005 (The Pile)

Max Activating Token Index: 183

Text #5

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0938. Min Act: -0.0000

Data Index: 1197043 (The Pile)

Max Activating Token Index: 358

Text #6

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0934. Min Act: -0.0000

Data Index: 1425992 (The Pile)

Max Activating Token Index: 330

Text #7

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0705. Min Act: -0.0000

Data Index: 248422 (The Pile)

Max Activating Token Index: 249

Text #8

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.2464. Min Act: -0.0000

Data Index: 604007 (The Pile)

Max Activating Token Index: 222

Text #9

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.7932. Min Act: -0.0000

Data Index: 304879 (The Pile)

Max Activating Token Index: 319

Text #10

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0873. Min Act: -0.0000

Data Index: 438648 (The Pile)

Max Activating Token Index: 814

Text #11

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0672. Min Act: -0.0000

Data Index: 1263487 (The Pile)

Max Activating Token Index: 445

Text #12

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8773. Min Act: -0.0000

Data Index: 1888538 (The Pile)

Max Activating Token Index: 121

Text #13

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.7407. Min Act: -0.0000

Data Index: 756982 (The Pile)

Max Activating Token Index: 160

Text #14

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8714. Min Act: -0.0000

Data Index: 1610851 (The Pile)

Max Activating Token Index: 314

Text #15

Max Range: 3.2464. Min Range: -3.2464

Max Act: 3.0093. Min Act: -0.0000

Data Index: 1711611 (The Pile)

Max Activating Token Index: 332

Text #16

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8976. Min Act: -0.0000

Data Index: 900047 (The Pile)

Max Activating Token Index: 128

Text #17

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8581. Min Act: -0.0000

Data Index: 1412114 (The Pile)

Max Activating Token Index: 807

Text #18

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.7151. Min Act: -0.0000

Data Index: 584480 (The Pile)

Max Activating Token Index: 636

Text #19

Max Range: 3.2464. Min Range: -3.2464

Max Act: 2.8980. Min Act: -0.0000

Data Index: 624565 (The Pile)

Max Activating Token Index: 237