Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3494 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0460. Min Act: -0.0000

Data Index: 1246330 (The Pile)

Max Activating Token Index: 873

Text #1

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0350. Min Act: -0.0000

Data Index: 1335087 (The Pile)

Max Activating Token Index: 1003

Text #2

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0389. Min Act: -0.0000

Data Index: 1459039 (The Pile)

Max Activating Token Index: 758

Text #3

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0405. Min Act: -0.0000

Data Index: 1680820 (The Pile)

Max Activating Token Index: 398

Text #4

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0415. Min Act: -0.0000

Data Index: 1940401 (The Pile)

Max Activating Token Index: 103

Text #5

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0345. Min Act: -0.0000

Data Index: 1033440 (The Pile)

Max Activating Token Index: 893

Text #6

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0326. Min Act: -0.0000

Data Index: 1893157 (The Pile)

Max Activating Token Index: 982

Text #7

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0335. Min Act: -0.0000

Data Index: 1794862 (The Pile)

Max Activating Token Index: 510

Text #8

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0392. Min Act: -0.0000

Data Index: 645132 (The Pile)

Max Activating Token Index: 942

Text #9

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0291. Min Act: -0.0000

Data Index: 322266 (The Pile)

Max Activating Token Index: 970

Text #10

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0338. Min Act: -0.0000

Data Index: 1452660 (The Pile)

Max Activating Token Index: 450

Text #11

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0312. Min Act: -0.0000

Data Index: 13290 (The Pile)

Max Activating Token Index: 963

Text #12

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0339. Min Act: -0.0000

Data Index: 1866574 (The Pile)

Max Activating Token Index: 895

Text #13

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0330. Min Act: -0.0000

Data Index: 1936828 (The Pile)

Max Activating Token Index: 1021

Text #14

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0298. Min Act: -0.0000

Data Index: 1044829 (The Pile)

Max Activating Token Index: 995

Text #15

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0333. Min Act: -0.0000

Data Index: 311607 (The Pile)

Max Activating Token Index: 67

Text #16

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0294. Min Act: -0.0000

Data Index: 248626 (The Pile)

Max Activating Token Index: 71

Text #17

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0279. Min Act: -0.0000

Data Index: 896012 (The Pile)

Max Activating Token Index: 93

Text #18

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0292. Min Act: -0.0000

Data Index: 386235 (The Pile)

Max Activating Token Index: 978

Text #19

Max Range: 0.0460. Min Range: -0.0460

Max Act: 0.0279. Min Act: -0.0000

Data Index: 1357240 (The Pile)

Max Activating Token Index: 714