Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2229 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.1990. Min Range: -4.1990

Max Act: 4.1990. Min Act: -0.0000

Data Index: 244361 (The Pile)

Max Activating Token Index: 276

Text #1

Max Range: 4.1990. Min Range: -4.1990

Max Act: 4.1757. Min Act: -0.0000

Data Index: 1378700 (The Pile)

Max Activating Token Index: 106

Text #2

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.4430. Min Act: -0.0000

Data Index: 934910 (The Pile)

Max Activating Token Index: 730

Text #3

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.7394. Min Act: -0.0000

Data Index: 568084 (The Pile)

Max Activating Token Index: 868

Text #4

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.3062. Min Act: -0.0000

Data Index: 1568032 (The Pile)

Max Activating Token Index: 610

Text #5

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.9135. Min Act: -0.0000

Data Index: 1616098 (The Pile)

Max Activating Token Index: 726

Text #6

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.2441. Min Act: -0.0000

Data Index: 234662 (The Pile)

Max Activating Token Index: 195

Text #7

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.2260. Min Act: -0.0000

Data Index: 88351 (The Pile)

Max Activating Token Index: 353

Text #8

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.0045. Min Act: -0.0000

Data Index: 1663893 (The Pile)

Max Activating Token Index: 543

Text #9

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.9956. Min Act: -0.0000

Data Index: 357801 (The Pile)

Max Activating Token Index: 209

Text #10

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.8197. Min Act: -0.0000

Data Index: 1476038 (The Pile)

Max Activating Token Index: 64

Text #11

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.7662. Min Act: -0.0000

Data Index: 1815191 (The Pile)

Max Activating Token Index: 621

Text #12

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.9360. Min Act: -0.0000

Data Index: 1322551 (The Pile)

Max Activating Token Index: 805

Text #13

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.7698. Min Act: -0.0000

Data Index: 1586443 (The Pile)

Max Activating Token Index: 145

Text #14

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.7154. Min Act: -0.0000

Data Index: 689789 (The Pile)

Max Activating Token Index: 348

Text #15

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.7337. Min Act: -0.0000

Data Index: 401495 (The Pile)

Max Activating Token Index: 792

Text #16

Max Range: 4.1990. Min Range: -4.1990

Max Act: 3.1555. Min Act: -0.0000

Data Index: 210875 (The Pile)

Max Activating Token Index: 695

Text #17

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.6595. Min Act: -0.0000

Data Index: 539602 (The Pile)

Max Activating Token Index: 408

Text #18

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.5168. Min Act: -0.0000

Data Index: 1243495 (The Pile)

Max Activating Token Index: 729

Text #19

Max Range: 4.1990. Min Range: -4.1990

Max Act: 2.8514. Min Act: -0.0000

Data Index: 1496319 (The Pile)

Max Activating Token Index: 493