Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2630 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7652. Min Act: -0.0000

Data Index: 666658 (The Pile)

Max Activating Token Index: 78

Text #1

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7146. Min Act: -0.0000

Data Index: 1191590 (The Pile)

Max Activating Token Index: 283

Text #2

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7019. Min Act: -0.0000

Data Index: 25888 (The Pile)

Max Activating Token Index: 500

Text #3

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7207. Min Act: -0.0000

Data Index: 669767 (The Pile)

Max Activating Token Index: 60

Text #4

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7306. Min Act: -0.0000

Data Index: 435952 (The Pile)

Max Activating Token Index: 206

Text #5

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.7021. Min Act: -0.0000

Data Index: 407250 (The Pile)

Max Activating Token Index: 222

Text #6

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6874. Min Act: -0.0000

Data Index: 1309615 (The Pile)

Max Activating Token Index: 326

Text #7

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6500. Min Act: -0.0000

Data Index: 1223852 (The Pile)

Max Activating Token Index: 109

Text #8

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6645. Min Act: -0.0000

Data Index: 1945239 (The Pile)

Max Activating Token Index: 532

Text #9

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6442. Min Act: -0.0000

Data Index: 319189 (The Pile)

Max Activating Token Index: 767

Text #10

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6079. Min Act: -0.0000

Data Index: 1202097 (The Pile)

Max Activating Token Index: 544

Text #11

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6084. Min Act: -0.0000

Data Index: 311426 (The Pile)

Max Activating Token Index: 134

Text #12

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5795. Min Act: -0.0000

Data Index: 1811856 (The Pile)

Max Activating Token Index: 862

Text #13

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.6083. Min Act: -0.0000

Data Index: 584364 (The Pile)

Max Activating Token Index: 130

Text #14

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5980. Min Act: -0.0000

Data Index: 339650 (The Pile)

Max Activating Token Index: 835

Text #15

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5760. Min Act: -0.0000

Data Index: 1249277 (The Pile)

Max Activating Token Index: 36

Text #16

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5811. Min Act: -0.0000

Data Index: 1238839 (The Pile)

Max Activating Token Index: 426

Text #17

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5641. Min Act: -0.0000

Data Index: 1004480 (The Pile)

Max Activating Token Index: 849

Text #18

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5653. Min Act: -0.0000

Data Index: 116769 (The Pile)

Max Activating Token Index: 230

Text #19

Max Range: 0.7652. Min Range: -0.7652

Max Act: 0.5422. Min Act: -0.0000

Data Index: 1123806 (The Pile)

Max Activating Token Index: 139