Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4133 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.4169. Min Act: -0.0000

Data Index: 922768 (The Pile)

Max Activating Token Index: 146

Text #1

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.0667. Min Act: -0.0000

Data Index: 1128257 (The Pile)

Max Activating Token Index: 648

Text #2

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.9959. Min Act: -0.0000

Data Index: 410737 (The Pile)

Max Activating Token Index: 638

Text #3

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2171. Min Act: -0.0000

Data Index: 283276 (The Pile)

Max Activating Token Index: 413

Text #4

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.9543. Min Act: -0.0000

Data Index: 321181 (The Pile)

Max Activating Token Index: 67

Text #5

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.9139. Min Act: -0.0000

Data Index: 1114902 (The Pile)

Max Activating Token Index: 323

Text #6

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.9112. Min Act: -0.0000

Data Index: 340930 (The Pile)

Max Activating Token Index: 876

Text #7

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.6517. Min Act: -0.0000

Data Index: 438008 (The Pile)

Max Activating Token Index: 681

Text #8

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.7669. Min Act: -0.0000

Data Index: 59389 (The Pile)

Max Activating Token Index: 527

Text #9

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.6401. Min Act: -0.0000

Data Index: 701173 (The Pile)

Max Activating Token Index: 493

Text #10

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8933. Min Act: -0.0000

Data Index: 1626307 (The Pile)

Max Activating Token Index: 830

Text #11

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.0072. Min Act: -0.0000

Data Index: 370086 (The Pile)

Max Activating Token Index: 87

Text #12

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.9955. Min Act: -0.0000

Data Index: 1346696 (The Pile)

Max Activating Token Index: 505

Text #13

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8524. Min Act: -0.0000

Data Index: 1986492 (The Pile)

Max Activating Token Index: 155

Text #14

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8510. Min Act: -0.0000

Data Index: 990569 (The Pile)

Max Activating Token Index: 727

Text #15

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.4625. Min Act: -0.0000

Data Index: 1312732 (The Pile)

Max Activating Token Index: 909

Text #16

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.7377. Min Act: -0.0000

Data Index: 1041928 (The Pile)

Max Activating Token Index: 475

Text #17

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8263. Min Act: -0.0000

Data Index: 1870331 (The Pile)

Max Activating Token Index: 894

Text #18

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8293. Min Act: -0.0000

Data Index: 1746024 (The Pile)

Max Activating Token Index: 318

Text #19

Max Range: 3.4169. Min Range: -3.4169

Max Act: 2.8283. Min Act: -0.0000

Data Index: 395002 (The Pile)

Max Activating Token Index: 604