Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2550 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.3143. Min Act: -0.0000

Data Index: 882963 (The Pile)

Max Activating Token Index: 1022

Text #1

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1268. Min Act: -0.0000

Data Index: 1099688 (The Pile)

Max Activating Token Index: 227

Text #2

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.2032. Min Act: -0.0000

Data Index: 1921698 (The Pile)

Max Activating Token Index: 459

Text #3

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0968. Min Act: -0.0000

Data Index: 960451 (The Pile)

Max Activating Token Index: 322

Text #4

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.8551. Min Act: -0.0000

Data Index: 1362422 (The Pile)

Max Activating Token Index: 538

Text #5

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.5338. Min Act: -0.0000

Data Index: 883402 (The Pile)

Max Activating Token Index: 799

Text #6

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.9566. Min Act: -0.0000

Data Index: 202086 (The Pile)

Max Activating Token Index: 464

Text #7

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.7367. Min Act: -0.0000

Data Index: 1118660 (The Pile)

Max Activating Token Index: 359

Text #8

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.8956. Min Act: -0.0000

Data Index: 41857 (The Pile)

Max Activating Token Index: 1007

Text #9

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.7486. Min Act: -0.0000

Data Index: 1582215 (The Pile)

Max Activating Token Index: 227

Text #10

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6183. Min Act: -0.0000

Data Index: 1772068 (The Pile)

Max Activating Token Index: 104

Text #11

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.8542. Min Act: -0.0000

Data Index: 962027 (The Pile)

Max Activating Token Index: 522

Text #12

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.5075. Min Act: -0.0000

Data Index: 53699 (The Pile)

Max Activating Token Index: 479

Text #13

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.8661. Min Act: -0.0000

Data Index: 16090 (The Pile)

Max Activating Token Index: 297

Text #14

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6897. Min Act: -0.0000

Data Index: 661920 (The Pile)

Max Activating Token Index: 160

Text #15

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6706. Min Act: -0.0000

Data Index: 1064036 (The Pile)

Max Activating Token Index: 1016

Text #16

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6106. Min Act: -0.0000

Data Index: 1374156 (The Pile)

Max Activating Token Index: 194

Text #17

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6500. Min Act: -0.0000

Data Index: 526305 (The Pile)

Max Activating Token Index: 463

Text #18

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6024. Min Act: -0.0000

Data Index: 1846894 (The Pile)

Max Activating Token Index: 75

Text #19

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.6057. Min Act: -0.0000

Data Index: 1654527 (The Pile)

Max Activating Token Index: 443