Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 454 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.1071. Min Act: -0.0000

Data Index: 1621904 (The Pile)

Max Activating Token Index: 893

Text #1

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.1015. Min Act: -0.0000

Data Index: 1397231 (The Pile)

Max Activating Token Index: 234

Text #2

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.9718. Min Act: -0.0000

Data Index: 562681 (The Pile)

Max Activating Token Index: 991

Text #3

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.1399. Min Act: -0.0000

Data Index: 74114 (The Pile)

Max Activating Token Index: 330

Text #4

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.9606. Min Act: -0.0000

Data Index: 1624402 (The Pile)

Max Activating Token Index: 189

Text #5

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.9329. Min Act: -0.0000

Data Index: 550157 (The Pile)

Max Activating Token Index: 368

Text #6

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.1051. Min Act: -0.0000

Data Index: 1316762 (The Pile)

Max Activating Token Index: 259

Text #7

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.0512. Min Act: -0.0000

Data Index: 292784 (The Pile)

Max Activating Token Index: 882

Text #8

Max Range: 4.1399. Min Range: -4.1399

Max Act: 4.0506. Min Act: -0.0000

Data Index: 1751479 (The Pile)

Max Activating Token Index: 972

Text #9

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.6775. Min Act: -0.0000

Data Index: 832506 (The Pile)

Max Activating Token Index: 644

Text #10

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.8299. Min Act: -0.0000

Data Index: 1535108 (The Pile)

Max Activating Token Index: 475

Text #11

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.8037. Min Act: -0.0000

Data Index: 265831 (The Pile)

Max Activating Token Index: 644

Text #12

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.6082. Min Act: -0.0000

Data Index: 1027937 (The Pile)

Max Activating Token Index: 692

Text #13

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.5836. Min Act: -0.0000

Data Index: 1473285 (The Pile)

Max Activating Token Index: 419

Text #14

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.5481. Min Act: -0.0000

Data Index: 609234 (The Pile)

Max Activating Token Index: 154

Text #15

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.7815. Min Act: -0.0000

Data Index: 251137 (The Pile)

Max Activating Token Index: 940

Text #16

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.6253. Min Act: -0.0000

Data Index: 554153 (The Pile)

Max Activating Token Index: 333

Text #17

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.4595. Min Act: -0.0000

Data Index: 237288 (The Pile)

Max Activating Token Index: 971

Text #18

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.4330. Min Act: -0.0000

Data Index: 1472211 (The Pile)

Max Activating Token Index: 704

Text #19

Max Range: 4.1399. Min Range: -4.1399

Max Act: 3.5760. Min Act: -0.0000

Data Index: 407477 (The Pile)

Max Activating Token Index: 595