Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1424 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 9.4818. Min Range: -9.4818

Max Act: 9.4818. Min Act: -0.0000

Data Index: 64593 (The Pile)

Max Activating Token Index: 601

Text #1

Max Range: 9.4818. Min Range: -9.4818

Max Act: 9.4641. Min Act: -0.0000

Data Index: 55448 (The Pile)

Max Activating Token Index: 42

Text #2

Max Range: 9.4818. Min Range: -9.4818

Max Act: 9.1056. Min Act: -0.0000

Data Index: 1762034 (The Pile)

Max Activating Token Index: 83

Text #3

Max Range: 9.4818. Min Range: -9.4818

Max Act: 9.2270. Min Act: -0.0000

Data Index: 387743 (The Pile)

Max Activating Token Index: 89

Text #4

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.2928. Min Act: -0.0000

Data Index: 1905336 (The Pile)

Max Activating Token Index: 218

Text #5

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.3929. Min Act: -0.0000

Data Index: 1342667 (The Pile)

Max Activating Token Index: 218

Text #6

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.6867. Min Act: -0.0000

Data Index: 1740025 (The Pile)

Max Activating Token Index: 42

Text #7

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.8118. Min Act: -0.0000

Data Index: 1649350 (The Pile)

Max Activating Token Index: 785

Text #8

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.6248. Min Act: -0.0000

Data Index: 806963 (The Pile)

Max Activating Token Index: 159

Text #9

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.3762. Min Act: -0.0000

Data Index: 1323973 (The Pile)

Max Activating Token Index: 363

Text #10

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.8357. Min Act: -0.0000

Data Index: 723183 (The Pile)

Max Activating Token Index: 513

Text #11

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.4768. Min Act: -0.0000

Data Index: 1669484 (The Pile)

Max Activating Token Index: 179

Text #12

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.1780. Min Act: -0.0000

Data Index: 1101751 (The Pile)

Max Activating Token Index: 546

Text #13

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.1087. Min Act: -0.0000

Data Index: 1695764 (The Pile)

Max Activating Token Index: 662

Text #14

Max Range: 9.4818. Min Range: -9.4818

Max Act: 7.5491. Min Act: -0.0000

Data Index: 863811 (The Pile)

Max Activating Token Index: 482

Text #15

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.3406. Min Act: -0.0000

Data Index: 1588736 (The Pile)

Max Activating Token Index: 70

Text #16

Max Range: 9.4818. Min Range: -9.4818

Max Act: 7.9002. Min Act: -0.0000

Data Index: 1368172 (The Pile)

Max Activating Token Index: 249

Text #17

Max Range: 9.4818. Min Range: -9.4818

Max Act: 8.8768. Min Act: -0.0000

Data Index: 1577126 (The Pile)

Max Activating Token Index: 687

Text #18

Max Range: 9.4818. Min Range: -9.4818

Max Act: 6.9266. Min Act: -0.0000

Data Index: 240191 (The Pile)

Max Activating Token Index: 943

Text #19

Max Range: 9.4818. Min Range: -9.4818

Max Act: 7.4661. Min Act: -0.0000

Data Index: 1508018 (The Pile)

Max Activating Token Index: 804