Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2951 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.2787. Min Act: -0.0002

Data Index: 87266 (C4 (Web Text))

Max Activating Token Index: 158

Text #1

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1795. Min Act: -0.0002

Data Index: 1045614 (C4 (Web Text))

Max Activating Token Index: 293

Text #2

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1625. Min Act: -0.0002

Data Index: 326132 (C4 (Web Text))

Max Activating Token Index: 183

Text #3

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1702. Min Act: -0.0002

Data Index: 415893 (C4 (Web Text))

Max Activating Token Index: 36

Text #4

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1339. Min Act: -0.0002

Data Index: 22490 (C4 (Web Text))

Max Activating Token Index: 617

Text #5

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1298. Min Act: -0.0002

Data Index: 968316 (C4 (Web Text))

Max Activating Token Index: 98

Text #6

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1223. Min Act: -0.0002

Data Index: 1235210 (C4 (Web Text))

Max Activating Token Index: 198

Text #7

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1330. Min Act: -0.0002

Data Index: 1209407 (C4 (Web Text))

Max Activating Token Index: 115

Text #8

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.1061. Min Act: -0.0002

Data Index: 349390 (C4 (Web Text))

Max Activating Token Index: 144

Text #9

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0901. Min Act: -0.0002

Data Index: 78909 (C4 (Web Text))

Max Activating Token Index: 742

Text #10

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0681. Min Act: -0.0003

Data Index: 80531 (C4 (Web Text))

Max Activating Token Index: 684

Text #11

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0708. Min Act: -0.0002

Data Index: 151384 (C4 (Web Text))

Max Activating Token Index: 141

Text #12

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0761. Min Act: -0.0002

Data Index: 304581 (C4 (Web Text))

Max Activating Token Index: 633

Text #13

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0465. Min Act: -0.0002

Data Index: 1255483 (C4 (Web Text))

Max Activating Token Index: 620

Text #14

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0527. Min Act: -0.0002

Data Index: 339306 (C4 (Web Text))

Max Activating Token Index: 131

Text #15

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0564. Min Act: -0.0002

Data Index: 870811 (C4 (Web Text))

Max Activating Token Index: 651

Text #16

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0373. Min Act: -0.0002

Data Index: 981226 (C4 (Web Text))

Max Activating Token Index: 373

Text #17

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0563. Min Act: -0.0002

Data Index: 1183715 (C4 (Web Text))

Max Activating Token Index: 104

Text #18

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0534. Min Act: -0.0002

Data Index: 508267 (C4 (Web Text))

Max Activating Token Index: 942

Text #19

Max Range: 1.2787. Min Range: -1.2787

Max Act: 1.0142. Min Act: -0.0002

Data Index: 1339388 (C4 (Web Text))

Max Activating Token Index: 684