Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 4012 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.4285. Min Act: -0.0000

Data Index: 1250378 (The Pile)

Max Activating Token Index: 217

Text #1

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.1534. Min Act: -0.0000

Data Index: 1641826 (The Pile)

Max Activating Token Index: 682

Text #2

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.3419. Min Act: -0.0000

Data Index: 1526109 (The Pile)

Max Activating Token Index: 971

Text #3

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.0904. Min Act: -0.0000

Data Index: 148869 (The Pile)

Max Activating Token Index: 411

Text #4

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.2924. Min Act: -0.0000

Data Index: 1405897 (The Pile)

Max Activating Token Index: 230

Text #5

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9490. Min Act: -0.0000

Data Index: 1054241 (The Pile)

Max Activating Token Index: 1003

Text #6

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.1390. Min Act: -0.0000

Data Index: 174953 (The Pile)

Max Activating Token Index: 310

Text #7

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.2510. Min Act: -0.0000

Data Index: 829373 (The Pile)

Max Activating Token Index: 855

Text #8

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.5300. Min Act: -0.0000

Data Index: 21128 (The Pile)

Max Activating Token Index: 53

Text #9

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9078. Min Act: -0.0000

Data Index: 772139 (The Pile)

Max Activating Token Index: 827

Text #10

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9653. Min Act: -0.0000

Data Index: 855274 (The Pile)

Max Activating Token Index: 165

Text #11

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9176. Min Act: -0.0000

Data Index: 558387 (The Pile)

Max Activating Token Index: 713

Text #12

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9400. Min Act: -0.0000

Data Index: 1516277 (The Pile)

Max Activating Token Index: 64

Text #13

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.9348. Min Act: -0.0000

Data Index: 1120003 (The Pile)

Max Activating Token Index: 585

Text #14

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.8145. Min Act: -0.0000

Data Index: 1155100 (The Pile)

Max Activating Token Index: 118

Text #15

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.1685. Min Act: -0.0000

Data Index: 1625032 (The Pile)

Max Activating Token Index: 157

Text #16

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.7960. Min Act: -0.0001

Data Index: 169060 (The Pile)

Max Activating Token Index: 462

Text #17

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.6997. Min Act: -0.0000

Data Index: 1662646 (The Pile)

Max Activating Token Index: 924

Text #18

Max Range: 3.5300. Min Range: -3.5300

Max Act: 2.6880. Min Act: -0.0000

Data Index: 787484 (The Pile)

Max Activating Token Index: 62

Text #19

Max Range: 3.5300. Min Range: -3.5300

Max Act: 3.0961. Min Act: -0.0000

Data Index: 1155092 (The Pile)

Max Activating Token Index: 287