Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 515 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.8207. Min Act: -0.0000

Data Index: 231442 (The Pile)

Max Activating Token Index: 394

Text #1

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.7146. Min Act: 0.0000

Data Index: 1394019 (The Pile)

Max Activating Token Index: 741

Text #2

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6954. Min Act: -0.0000

Data Index: 1661200 (The Pile)

Max Activating Token Index: 621

Text #3

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.7003. Min Act: -0.0000

Data Index: 1562816 (The Pile)

Max Activating Token Index: 1011

Text #4

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6813. Min Act: 0.0000

Data Index: 1254091 (The Pile)

Max Activating Token Index: 972

Text #5

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6991. Min Act: -0.0000

Data Index: 668032 (The Pile)

Max Activating Token Index: 389

Text #6

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6301. Min Act: -0.0000

Data Index: 325416 (The Pile)

Max Activating Token Index: 517

Text #7

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5662. Min Act: -0.0000

Data Index: 268203 (The Pile)

Max Activating Token Index: 973

Text #8

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6618. Min Act: -0.0000

Data Index: 429958 (The Pile)

Max Activating Token Index: 403

Text #9

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6590. Min Act: -0.0000

Data Index: 259442 (The Pile)

Max Activating Token Index: 612

Text #10

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6270. Min Act: -0.0000

Data Index: 1103466 (The Pile)

Max Activating Token Index: 912

Text #11

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5443. Min Act: -0.0000

Data Index: 1936648 (The Pile)

Max Activating Token Index: 569

Text #12

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6186. Min Act: -0.0000

Data Index: 746670 (The Pile)

Max Activating Token Index: 819

Text #13

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5855. Min Act: -0.0000

Data Index: 826889 (The Pile)

Max Activating Token Index: 818

Text #14

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6347. Min Act: -0.0000

Data Index: 17357 (The Pile)

Max Activating Token Index: 382

Text #15

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5909. Min Act: -0.0000

Data Index: 561415 (The Pile)

Max Activating Token Index: 476

Text #16

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.6184. Min Act: -0.0000

Data Index: 1463328 (The Pile)

Max Activating Token Index: 203

Text #17

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5931. Min Act: 0.0000

Data Index: 1165608 (The Pile)

Max Activating Token Index: 1021

Text #18

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5004. Min Act: -0.0000

Data Index: 1277049 (The Pile)

Max Activating Token Index: 42

Text #19

Max Range: 0.8207. Min Range: -0.8207

Max Act: 0.5144. Min Act: -0.0000

Data Index: 1786942 (The Pile)

Max Activating Token Index: 156