Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2408 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0557. Min Range: -1.0557

Max Act: 1.0557. Min Act: -0.0000

Data Index: 1289038 (The Pile)

Max Activating Token Index: 964

Text #1

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.9998. Min Act: -0.0000

Data Index: 1293713 (The Pile)

Max Activating Token Index: 969

Text #2

Max Range: 1.0557. Min Range: -1.0557

Max Act: 1.0364. Min Act: -0.0000

Data Index: 352863 (The Pile)

Max Activating Token Index: 983

Text #3

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8268. Min Act: -0.0000

Data Index: 1183114 (The Pile)

Max Activating Token Index: 53

Text #4

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.9379. Min Act: -0.0000

Data Index: 1883433 (The Pile)

Max Activating Token Index: 331

Text #5

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8931. Min Act: -0.0000

Data Index: 381539 (The Pile)

Max Activating Token Index: 535

Text #6

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8902. Min Act: -0.0000

Data Index: 1748600 (The Pile)

Max Activating Token Index: 606

Text #7

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8111. Min Act: -0.0000

Data Index: 766693 (The Pile)

Max Activating Token Index: 204

Text #8

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.7749. Min Act: -0.0000

Data Index: 1541447 (The Pile)

Max Activating Token Index: 818

Text #9

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8852. Min Act: -0.0000

Data Index: 1773688 (The Pile)

Max Activating Token Index: 768

Text #10

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8244. Min Act: -0.0000

Data Index: 983129 (The Pile)

Max Activating Token Index: 511

Text #11

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.7573. Min Act: -0.0000

Data Index: 1428563 (The Pile)

Max Activating Token Index: 49

Text #12

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.8019. Min Act: -0.0000

Data Index: 1171636 (The Pile)

Max Activating Token Index: 167

Text #13

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.7509. Min Act: -0.0000

Data Index: 1988486 (The Pile)

Max Activating Token Index: 872

Text #14

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.7580. Min Act: -0.0000

Data Index: 1498809 (The Pile)

Max Activating Token Index: 1022

Text #15

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.6849. Min Act: -0.0000

Data Index: 1472243 (The Pile)

Max Activating Token Index: 206

Text #16

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.7735. Min Act: -0.0000

Data Index: 1214283 (The Pile)

Max Activating Token Index: 929

Text #17

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.6677. Min Act: -0.0000

Data Index: 108266 (The Pile)

Max Activating Token Index: 661

Text #18

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.6845. Min Act: -0.0000

Data Index: 163655 (The Pile)

Max Activating Token Index: 720

Text #19

Max Range: 1.0557. Min Range: -1.0557

Max Act: 0.6661. Min Act: -0.0000

Data Index: 399124 (The Pile)

Max Activating Token Index: 305