Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4106 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.4939. Min Act: -0.0000

Data Index: 1281730 (The Pile)

Max Activating Token Index: 323

Text #1

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.4389. Min Act: -0.0000

Data Index: 1365372 (The Pile)

Max Activating Token Index: 617

Text #2

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.4034. Min Act: -0.0000

Data Index: 699134 (The Pile)

Max Activating Token Index: 651

Text #3

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3791. Min Act: -0.0000

Data Index: 1518293 (The Pile)

Max Activating Token Index: 711

Text #4

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3748. Min Act: -0.0000

Data Index: 1576499 (The Pile)

Max Activating Token Index: 12

Text #5

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3678. Min Act: -0.0000

Data Index: 1582657 (The Pile)

Max Activating Token Index: 770

Text #6

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3857. Min Act: -0.0000

Data Index: 1495040 (The Pile)

Max Activating Token Index: 705

Text #7

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3307. Min Act: -0.0000

Data Index: 2300 (The Pile)

Max Activating Token Index: 712

Text #8

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3320. Min Act: -0.0000

Data Index: 600762 (The Pile)

Max Activating Token Index: 454

Text #9

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3446. Min Act: -0.0000

Data Index: 1628824 (The Pile)

Max Activating Token Index: 991

Text #10

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3337. Min Act: -0.0000

Data Index: 1261565 (The Pile)

Max Activating Token Index: 552

Text #11

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3333. Min Act: -0.0000

Data Index: 252096 (The Pile)

Max Activating Token Index: 539

Text #12

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3217. Min Act: -0.0000

Data Index: 1805513 (The Pile)

Max Activating Token Index: 612

Text #13

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3452. Min Act: -0.0000

Data Index: 763851 (The Pile)

Max Activating Token Index: 862

Text #14

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3311. Min Act: -0.0000

Data Index: 1222212 (The Pile)

Max Activating Token Index: 711

Text #15

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3156. Min Act: -0.0000

Data Index: 1217118 (The Pile)

Max Activating Token Index: 869

Text #16

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3173. Min Act: -0.0000

Data Index: 1283436 (The Pile)

Max Activating Token Index: 132

Text #17

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3380. Min Act: -0.0000

Data Index: 578915 (The Pile)

Max Activating Token Index: 973

Text #18

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3247. Min Act: -0.0000

Data Index: 1637180 (The Pile)

Max Activating Token Index: 951

Text #19

Max Range: 0.4939. Min Range: -0.4939

Max Act: 0.3010. Min Act: -0.0000

Data Index: 1255624 (The Pile)

Max Activating Token Index: 116