Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2409 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.7157. Min Act: -0.0000

Data Index: 1663960 (The Pile)

Max Activating Token Index: 272

Text #1

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.5143. Min Act: -0.0000

Data Index: 1699407 (The Pile)

Max Activating Token Index: 604

Text #2

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.1606. Min Act: -0.0000

Data Index: 65363 (The Pile)

Max Activating Token Index: 697

Text #3

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.2382. Min Act: -0.0000

Data Index: 1458206 (The Pile)

Max Activating Token Index: 508

Text #4

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.5383. Min Act: -0.0000

Data Index: 1508534 (The Pile)

Max Activating Token Index: 430

Text #5

Max Range: 3.7157. Min Range: -3.7157

Max Act: 3.1033. Min Act: -0.0000

Data Index: 728540 (The Pile)

Max Activating Token Index: 910

Text #6

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.8006. Min Act: -0.0000

Data Index: 309089 (The Pile)

Max Activating Token Index: 994

Text #7

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.8033. Min Act: -0.0000

Data Index: 1254762 (The Pile)

Max Activating Token Index: 988

Text #8

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.6955. Min Act: -0.0000

Data Index: 34756 (The Pile)

Max Activating Token Index: 983

Text #9

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.9853. Min Act: -0.0000

Data Index: 1257123 (The Pile)

Max Activating Token Index: 308

Text #10

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7348. Min Act: -0.0000

Data Index: 161821 (The Pile)

Max Activating Token Index: 758

Text #11

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7386. Min Act: -0.0000

Data Index: 1153908 (The Pile)

Max Activating Token Index: 427

Text #12

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7288. Min Act: -0.0000

Data Index: 420367 (The Pile)

Max Activating Token Index: 389

Text #13

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.3214. Min Act: -0.0000

Data Index: 856260 (The Pile)

Max Activating Token Index: 212

Text #14

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7370. Min Act: -0.0000

Data Index: 153357 (The Pile)

Max Activating Token Index: 934

Text #15

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7498. Min Act: -0.0000

Data Index: 1410253 (The Pile)

Max Activating Token Index: 69

Text #16

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7100. Min Act: -0.0000

Data Index: 1924049 (The Pile)

Max Activating Token Index: 655

Text #17

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7490. Min Act: -0.0000

Data Index: 1726287 (The Pile)

Max Activating Token Index: 855

Text #18

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.5710. Min Act: -0.0000

Data Index: 264317 (The Pile)

Max Activating Token Index: 278

Text #19

Max Range: 3.7157. Min Range: -3.7157

Max Act: 2.7142. Min Act: -0.0000

Data Index: 1750843 (The Pile)

Max Activating Token Index: 68