Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2565 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.8612. Min Act: -0.0000

Data Index: 875340 (The Pile)

Max Activating Token Index: 893

Text #1

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6487. Min Act: -0.0000

Data Index: 1428713 (The Pile)

Max Activating Token Index: 32

Text #2

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6736. Min Act: -0.0000

Data Index: 266600 (The Pile)

Max Activating Token Index: 351

Text #3

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6267. Min Act: -0.0000

Data Index: 661016 (The Pile)

Max Activating Token Index: 924

Text #4

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5921. Min Act: -0.0000

Data Index: 554221 (The Pile)

Max Activating Token Index: 626

Text #5

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6072. Min Act: -0.0000

Data Index: 965817 (The Pile)

Max Activating Token Index: 731

Text #6

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6177. Min Act: -0.0000

Data Index: 159284 (The Pile)

Max Activating Token Index: 462

Text #7

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5941. Min Act: -0.0000

Data Index: 1257154 (The Pile)

Max Activating Token Index: 953

Text #8

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5865. Min Act: -0.0000

Data Index: 249057 (The Pile)

Max Activating Token Index: 64

Text #9

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5165. Min Act: -0.0000

Data Index: 1645414 (The Pile)

Max Activating Token Index: 1003

Text #10

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5537. Min Act: -0.0000

Data Index: 1131129 (The Pile)

Max Activating Token Index: 521

Text #11

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5897. Min Act: -0.0000

Data Index: 845437 (The Pile)

Max Activating Token Index: 258

Text #12

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6043. Min Act: -0.0000

Data Index: 1454491 (The Pile)

Max Activating Token Index: 50

Text #13

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6061. Min Act: -0.0000

Data Index: 367672 (The Pile)

Max Activating Token Index: 667

Text #14

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.6015. Min Act: -0.0000

Data Index: 292613 (The Pile)

Max Activating Token Index: 46

Text #15

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5789. Min Act: -0.0000

Data Index: 1816394 (The Pile)

Max Activating Token Index: 402

Text #16

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5500. Min Act: -0.0000

Data Index: 532472 (The Pile)

Max Activating Token Index: 273

Text #17

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5494. Min Act: -0.0000

Data Index: 367299 (The Pile)

Max Activating Token Index: 168

Text #18

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5641. Min Act: -0.0000

Data Index: 687794 (The Pile)

Max Activating Token Index: 224

Text #19

Max Range: 0.8612. Min Range: -0.8612

Max Act: 0.5707. Min Act: -0.0000

Data Index: 1367751 (The Pile)

Max Activating Token Index: 623