Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2239 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1126. Min Range: -3.1126

Max Act: 3.1126. Min Act: -0.0000

Data Index: 1250519 (The Pile)

Max Activating Token Index: 622

Text #1

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.8435. Min Act: -0.0000

Data Index: 397880 (The Pile)

Max Activating Token Index: 127

Text #2

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.8091. Min Act: -0.0000

Data Index: 1809551 (The Pile)

Max Activating Token Index: 653

Text #3

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.9667. Min Act: -0.0000

Data Index: 23011 (The Pile)

Max Activating Token Index: 195

Text #4

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.6044. Min Act: -0.0000

Data Index: 1562189 (The Pile)

Max Activating Token Index: 59

Text #5

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.5281. Min Act: -0.0000

Data Index: 859692 (The Pile)

Max Activating Token Index: 65

Text #6

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.5169. Min Act: -0.0000

Data Index: 434940 (The Pile)

Max Activating Token Index: 528

Text #7

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.5170. Min Act: -0.0000

Data Index: 389885 (The Pile)

Max Activating Token Index: 650

Text #8

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.4741. Min Act: -0.0000

Data Index: 1865432 (The Pile)

Max Activating Token Index: 512

Text #9

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.3292. Min Act: -0.0000

Data Index: 1514883 (The Pile)

Max Activating Token Index: 399

Text #10

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.2969. Min Act: -0.0000

Data Index: 1150681 (The Pile)

Max Activating Token Index: 945

Text #11

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.2933. Min Act: -0.0000

Data Index: 609370 (The Pile)

Max Activating Token Index: 635

Text #12

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.4110. Min Act: -0.0000

Data Index: 297779 (The Pile)

Max Activating Token Index: 101

Text #13

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.4282. Min Act: -0.0000

Data Index: 1299302 (The Pile)

Max Activating Token Index: 876

Text #14

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.1943. Min Act: -0.0000

Data Index: 1433632 (The Pile)

Max Activating Token Index: 985

Text #15

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.3111. Min Act: -0.0000

Data Index: 1162552 (The Pile)

Max Activating Token Index: 763

Text #16

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.3047. Min Act: -0.0000

Data Index: 1658736 (The Pile)

Max Activating Token Index: 41

Text #17

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.2922. Min Act: -0.0000

Data Index: 275127 (The Pile)

Max Activating Token Index: 76

Text #18

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.2669. Min Act: -0.0000

Data Index: 1789966 (The Pile)

Max Activating Token Index: 258

Text #19

Max Range: 3.1126. Min Range: -3.1126

Max Act: 2.1280. Min Act: -0.0000

Data Index: 1283995 (The Pile)

Max Activating Token Index: 168