Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3086 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.5361. Min Act: -0.0000

Data Index: 1856508 (The Pile)

Max Activating Token Index: 239

Text #1

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.3016. Min Act: -0.0000

Data Index: 1216772 (The Pile)

Max Activating Token Index: 88

Text #2

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2399. Min Act: -0.0000

Data Index: 1207989 (The Pile)

Max Activating Token Index: 807

Text #3

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.3078. Min Act: -0.0000

Data Index: 900523 (The Pile)

Max Activating Token Index: 1009

Text #4

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2970. Min Act: -0.0000

Data Index: 1685828 (The Pile)

Max Activating Token Index: 126

Text #5

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2845. Min Act: -0.0000

Data Index: 974254 (The Pile)

Max Activating Token Index: 460

Text #6

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1772. Min Act: -0.0000

Data Index: 1600355 (The Pile)

Max Activating Token Index: 935

Text #7

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2246. Min Act: -0.0000

Data Index: 829525 (The Pile)

Max Activating Token Index: 92

Text #8

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1485. Min Act: -0.0000

Data Index: 503797 (The Pile)

Max Activating Token Index: 1016

Text #9

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2090. Min Act: -0.0000

Data Index: 1036149 (The Pile)

Max Activating Token Index: 504

Text #10

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2351. Min Act: -0.0000

Data Index: 1834463 (The Pile)

Max Activating Token Index: 840

Text #11

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1736. Min Act: -0.0000

Data Index: 1374478 (The Pile)

Max Activating Token Index: 696

Text #12

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1994. Min Act: -0.0000

Data Index: 1519972 (The Pile)

Max Activating Token Index: 100

Text #13

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1623. Min Act: -0.0000

Data Index: 1854669 (The Pile)

Max Activating Token Index: 150

Text #14

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1862. Min Act: -0.0000

Data Index: 510167 (The Pile)

Max Activating Token Index: 782

Text #15

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1713. Min Act: -0.0000

Data Index: 554830 (The Pile)

Max Activating Token Index: 1019

Text #16

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1445. Min Act: -0.0000

Data Index: 769174 (The Pile)

Max Activating Token Index: 970

Text #17

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1623. Min Act: -0.0000

Data Index: 1219377 (The Pile)

Max Activating Token Index: 362

Text #18

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.1686. Min Act: -0.0000

Data Index: 469525 (The Pile)

Max Activating Token Index: 371

Text #19

Max Range: 1.5361. Min Range: -1.5361

Max Act: 1.2603. Min Act: -0.0000

Data Index: 1516593 (The Pile)

Max Activating Token Index: 534