Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2220 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.9183. Min Act: -0.0000

Data Index: 430359 (The Pile)

Max Activating Token Index: 695

Text #1

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.8168. Min Act: -0.0000

Data Index: 1914443 (The Pile)

Max Activating Token Index: 867

Text #2

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.8311. Min Act: -0.0000

Data Index: 1969502 (The Pile)

Max Activating Token Index: 751

Text #3

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7959. Min Act: -0.0000

Data Index: 369367 (The Pile)

Max Activating Token Index: 183

Text #4

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.8011. Min Act: -0.0000

Data Index: 468360 (The Pile)

Max Activating Token Index: 520

Text #5

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7537. Min Act: -0.0000

Data Index: 1645554 (The Pile)

Max Activating Token Index: 375

Text #6

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7317. Min Act: -0.0000

Data Index: 522416 (The Pile)

Max Activating Token Index: 634

Text #7

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7976. Min Act: -0.0000

Data Index: 1129072 (The Pile)

Max Activating Token Index: 876

Text #8

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7075. Min Act: -0.0000

Data Index: 426136 (The Pile)

Max Activating Token Index: 817

Text #9

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7715. Min Act: -0.0000

Data Index: 1370615 (The Pile)

Max Activating Token Index: 335

Text #10

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7574. Min Act: -0.0000

Data Index: 1444494 (The Pile)

Max Activating Token Index: 409

Text #11

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7717. Min Act: -0.0000

Data Index: 867969 (The Pile)

Max Activating Token Index: 94

Text #12

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6314. Min Act: -0.0000

Data Index: 299553 (The Pile)

Max Activating Token Index: 746

Text #13

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7079. Min Act: -0.0000

Data Index: 684925 (The Pile)

Max Activating Token Index: 373

Text #14

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6507. Min Act: -0.0000

Data Index: 1198676 (The Pile)

Max Activating Token Index: 862

Text #15

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6841. Min Act: -0.0000

Data Index: 1355120 (The Pile)

Max Activating Token Index: 216

Text #16

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.7028. Min Act: -0.0000

Data Index: 259085 (The Pile)

Max Activating Token Index: 691

Text #17

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6727. Min Act: -0.0000

Data Index: 583867 (The Pile)

Max Activating Token Index: 676

Text #18

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6790. Min Act: -0.0000

Data Index: 1138427 (The Pile)

Max Activating Token Index: 831

Text #19

Max Range: 0.9183. Min Range: -0.9183

Max Act: 0.6734. Min Act: -0.0000

Data Index: 1661477 (The Pile)

Max Activating Token Index: 630