Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1105 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.6139. Min Act: -0.0000

Data Index: 510493 (The Pile)

Max Activating Token Index: 671

Text #1

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.6328. Min Act: -0.0000

Data Index: 1409919 (The Pile)

Max Activating Token Index: 179

Text #2

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.6264. Min Act: -0.0000

Data Index: 403528 (The Pile)

Max Activating Token Index: 334

Text #3

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.6048. Min Act: -0.0000

Data Index: 765772 (The Pile)

Max Activating Token Index: 324

Text #4

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5822. Min Act: -0.0000

Data Index: 354960 (The Pile)

Max Activating Token Index: 122

Text #5

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5749. Min Act: -0.0000

Data Index: 1146992 (The Pile)

Max Activating Token Index: 66

Text #6

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.6475. Min Act: -0.0000

Data Index: 1419084 (The Pile)

Max Activating Token Index: 648

Text #7

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5767. Min Act: -0.0000

Data Index: 1436070 (The Pile)

Max Activating Token Index: 589

Text #8

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5580. Min Act: -0.0000

Data Index: 72077 (The Pile)

Max Activating Token Index: 670

Text #9

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5190. Min Act: -0.0000

Data Index: 594559 (The Pile)

Max Activating Token Index: 612

Text #10

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5516. Min Act: -0.0000

Data Index: 1694300 (The Pile)

Max Activating Token Index: 472

Text #11

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5012. Min Act: -0.0000

Data Index: 915366 (The Pile)

Max Activating Token Index: 634

Text #12

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5512. Min Act: -0.0000

Data Index: 1278097 (The Pile)

Max Activating Token Index: 298

Text #13

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5254. Min Act: -0.0000

Data Index: 263910 (The Pile)

Max Activating Token Index: 863

Text #14

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5416. Min Act: -0.0000

Data Index: 1740846 (The Pile)

Max Activating Token Index: 51

Text #15

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5340. Min Act: -0.0000

Data Index: 1061669 (The Pile)

Max Activating Token Index: 661

Text #16

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5390. Min Act: -0.0000

Data Index: 1643730 (The Pile)

Max Activating Token Index: 999

Text #17

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5199. Min Act: -0.0000

Data Index: 1822168 (The Pile)

Max Activating Token Index: 249

Text #18

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5107. Min Act: -0.0000

Data Index: 1723000 (The Pile)

Max Activating Token Index: 102

Text #19

Max Range: 0.6475. Min Range: -0.6475

Max Act: 0.5984. Min Act: -0.0000

Data Index: 247583 (The Pile)

Max Activating Token Index: 457