Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 132 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.5355. Min Act: -0.0000

Data Index: 759344 (The Pile)

Max Activating Token Index: 418

Text #1

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.5074. Min Act: -0.0000

Data Index: 1638656 (The Pile)

Max Activating Token Index: 751

Text #2

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.4811. Min Act: -0.0000

Data Index: 1309084 (The Pile)

Max Activating Token Index: 86

Text #3

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.5133. Min Act: -0.0000

Data Index: 1575358 (The Pile)

Max Activating Token Index: 992

Text #4

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.4772. Min Act: -0.0000

Data Index: 305819 (The Pile)

Max Activating Token Index: 329

Text #5

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.4741. Min Act: -0.0000

Data Index: 921120 (The Pile)

Max Activating Token Index: 1020

Text #6

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3729. Min Act: -0.0000

Data Index: 1567604 (The Pile)

Max Activating Token Index: 256

Text #7

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3849. Min Act: -0.0000

Data Index: 1377734 (The Pile)

Max Activating Token Index: 365

Text #8

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.4333. Min Act: -0.0000

Data Index: 627209 (The Pile)

Max Activating Token Index: 494

Text #9

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3942. Min Act: -0.0000

Data Index: 1881679 (The Pile)

Max Activating Token Index: 352

Text #10

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3829. Min Act: -0.0000

Data Index: 404804 (The Pile)

Max Activating Token Index: 277

Text #11

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3799. Min Act: -0.0000

Data Index: 469157 (The Pile)

Max Activating Token Index: 77

Text #12

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3898. Min Act: -0.0000

Data Index: 925489 (The Pile)

Max Activating Token Index: 319

Text #13

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3654. Min Act: -0.0000

Data Index: 1505914 (The Pile)

Max Activating Token Index: 864

Text #14

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.4070. Min Act: -0.0000

Data Index: 1031264 (The Pile)

Max Activating Token Index: 178

Text #15

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3564. Min Act: -0.0000

Data Index: 1192741 (The Pile)

Max Activating Token Index: 311

Text #16

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3702. Min Act: -0.0000

Data Index: 144711 (The Pile)

Max Activating Token Index: 291

Text #17

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3659. Min Act: -0.0000

Data Index: 1569399 (The Pile)

Max Activating Token Index: 610

Text #18

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3737. Min Act: -0.0000

Data Index: 581085 (The Pile)

Max Activating Token Index: 535

Text #19

Max Range: 0.5355. Min Range: -0.5355

Max Act: 0.3283. Min Act: -0.0000

Data Index: 1495970 (The Pile)

Max Activating Token Index: 304