Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3644 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.5808. Min Act: -0.0000

Data Index: 896915 (The Pile)

Max Activating Token Index: 362

Text #1

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.3541. Min Act: -0.0000

Data Index: 1005235 (The Pile)

Max Activating Token Index: 600

Text #2

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.1681. Min Act: -0.0000

Data Index: 1780131 (The Pile)

Max Activating Token Index: 291

Text #3

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.0284. Min Act: -0.0000

Data Index: 1008040 (The Pile)

Max Activating Token Index: 227

Text #4

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.1347. Min Act: -0.0000

Data Index: 1304719 (The Pile)

Max Activating Token Index: 692

Text #5

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.3309. Min Act: -0.0000

Data Index: 563810 (The Pile)

Max Activating Token Index: 190

Text #6

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.1132. Min Act: -0.0000

Data Index: 1142423 (The Pile)

Max Activating Token Index: 201

Text #7

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.0564. Min Act: -0.0000

Data Index: 1860825 (The Pile)

Max Activating Token Index: 891

Text #8

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.8469. Min Act: -0.0000

Data Index: 386575 (The Pile)

Max Activating Token Index: 592

Text #9

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.9753. Min Act: -0.0000

Data Index: 1013201 (The Pile)

Max Activating Token Index: 58

Text #10

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.0945. Min Act: -0.0000

Data Index: 816565 (The Pile)

Max Activating Token Index: 79

Text #11

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.8876. Min Act: -0.0000

Data Index: 1129764 (The Pile)

Max Activating Token Index: 561

Text #12

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.0167. Min Act: -0.0000

Data Index: 1288391 (The Pile)

Max Activating Token Index: 34

Text #13

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.8329. Min Act: -0.0000

Data Index: 1376200 (The Pile)

Max Activating Token Index: 73

Text #14

Max Range: 3.5808. Min Range: -3.5808

Max Act: 3.0042. Min Act: -0.0000

Data Index: 1664171 (The Pile)

Max Activating Token Index: 1022

Text #15

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.8111. Min Act: -0.0000

Data Index: 1374915 (The Pile)

Max Activating Token Index: 190

Text #16

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.7660. Min Act: -0.0000

Data Index: 825004 (The Pile)

Max Activating Token Index: 414

Text #17

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.8971. Min Act: -0.0000

Data Index: 1442221 (The Pile)

Max Activating Token Index: 288

Text #18

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.7837. Min Act: -0.0000

Data Index: 1972667 (The Pile)

Max Activating Token Index: 857

Text #19

Max Range: 3.5808. Min Range: -3.5808

Max Act: 2.5976. Min Act: -0.0000

Data Index: 1602406 (The Pile)

Max Activating Token Index: 739