Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1132 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3815. Min Act: -0.0000

Data Index: 1558442 (The Pile)

Max Activating Token Index: 820

Text #1

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3805. Min Act: -0.0000

Data Index: 528053 (The Pile)

Max Activating Token Index: 343

Text #2

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3544. Min Act: -0.0000

Data Index: 1646979 (The Pile)

Max Activating Token Index: 658

Text #3

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3787. Min Act: -0.0000

Data Index: 836782 (The Pile)

Max Activating Token Index: 865

Text #4

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3209. Min Act: -0.0000

Data Index: 601493 (The Pile)

Max Activating Token Index: 823

Text #5

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3385. Min Act: -0.0000

Data Index: 1871632 (The Pile)

Max Activating Token Index: 617

Text #6

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3220. Min Act: -0.0000

Data Index: 1219653 (The Pile)

Max Activating Token Index: 577

Text #7

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3081. Min Act: -0.0000

Data Index: 269808 (The Pile)

Max Activating Token Index: 873

Text #8

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3187. Min Act: -0.0000

Data Index: 837648 (The Pile)

Max Activating Token Index: 408

Text #9

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3241. Min Act: -0.0000

Data Index: 1027438 (The Pile)

Max Activating Token Index: 73

Text #10

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3078. Min Act: -0.0000

Data Index: 1374284 (The Pile)

Max Activating Token Index: 459

Text #11

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2914. Min Act: -0.0000

Data Index: 558588 (The Pile)

Max Activating Token Index: 265

Text #12

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2914. Min Act: -0.0000

Data Index: 436423 (The Pile)

Max Activating Token Index: 356

Text #13

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.3225. Min Act: -0.0000

Data Index: 1164738 (The Pile)

Max Activating Token Index: 620

Text #14

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2905. Min Act: -0.0000

Data Index: 106761 (The Pile)

Max Activating Token Index: 447

Text #15

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2798. Min Act: -0.0000

Data Index: 1312658 (The Pile)

Max Activating Token Index: 823

Text #16

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2832. Min Act: -0.0000

Data Index: 1246658 (The Pile)

Max Activating Token Index: 419

Text #17

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2834. Min Act: -0.0000

Data Index: 238523 (The Pile)

Max Activating Token Index: 234

Text #18

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2701. Min Act: -0.0000

Data Index: 1148742 (The Pile)

Max Activating Token Index: 381

Text #19

Max Range: 0.3815. Min Range: -0.3815

Max Act: 0.2659. Min Act: -0.0000

Data Index: 841419 (The Pile)

Max Activating Token Index: 335