Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2082 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.7184. Min Act: -0.0000

Data Index: 121070 (The Pile)

Max Activating Token Index: 472

Text #1

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.7159. Min Act: -0.0000

Data Index: 201737 (The Pile)

Max Activating Token Index: 132

Text #2

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.7043. Min Act: -0.0000

Data Index: 1791403 (The Pile)

Max Activating Token Index: 481

Text #3

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.7000. Min Act: -0.0000

Data Index: 1292917 (The Pile)

Max Activating Token Index: 73

Text #4

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6851. Min Act: -0.0000

Data Index: 1382799 (The Pile)

Max Activating Token Index: 716

Text #5

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6439. Min Act: -0.0000

Data Index: 1223701 (The Pile)

Max Activating Token Index: 59

Text #6

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6967. Min Act: -0.0000

Data Index: 1437731 (The Pile)

Max Activating Token Index: 926

Text #7

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6571. Min Act: -0.0000

Data Index: 463832 (The Pile)

Max Activating Token Index: 1000

Text #8

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6434. Min Act: -0.0000

Data Index: 1782371 (The Pile)

Max Activating Token Index: 82

Text #9

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5656. Min Act: -0.0000

Data Index: 1795782 (The Pile)

Max Activating Token Index: 659

Text #10

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.6102. Min Act: -0.0000

Data Index: 1672707 (The Pile)

Max Activating Token Index: 205

Text #11

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5791. Min Act: -0.0000

Data Index: 613110 (The Pile)

Max Activating Token Index: 900

Text #12

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5878. Min Act: -0.0000

Data Index: 1498335 (The Pile)

Max Activating Token Index: 212

Text #13

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5998. Min Act: -0.0000

Data Index: 1790669 (The Pile)

Max Activating Token Index: 348

Text #14

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5956. Min Act: -0.0000

Data Index: 1972011 (The Pile)

Max Activating Token Index: 683

Text #15

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5729. Min Act: -0.0000

Data Index: 690020 (The Pile)

Max Activating Token Index: 412

Text #16

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5981. Min Act: -0.0000

Data Index: 1410033 (The Pile)

Max Activating Token Index: 538

Text #17

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5252. Min Act: -0.0000

Data Index: 1173632 (The Pile)

Max Activating Token Index: 767

Text #18

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5557. Min Act: -0.0000

Data Index: 661487 (The Pile)

Max Activating Token Index: 192

Text #19

Max Range: 0.7184. Min Range: -0.7184

Max Act: 0.5687. Min Act: -0.0000

Data Index: 1234744 (The Pile)

Max Activating Token Index: 125