Model: SoLU Model: 6 Layers, 3072 Neurons per Layer

Dataset: The Pile

Neuron 2593 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-6l-pile')

Text #0

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.5335. Min Act: -0.0001

Data Index: 38469 (The Pile)

Max Activating Token Index: 933

Text #1

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4949. Min Act: -0.0000

Data Index: 1228256 (The Pile)

Max Activating Token Index: 871

Text #2

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4656. Min Act: -0.0001

Data Index: 995455 (The Pile)

Max Activating Token Index: 936

Text #3

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4458. Min Act: -0.0001

Data Index: 906738 (The Pile)

Max Activating Token Index: 647

Text #4

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4377. Min Act: -0.0001

Data Index: 247382 (The Pile)

Max Activating Token Index: 769

Text #5

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4480. Min Act: -0.0001

Data Index: 1538073 (The Pile)

Max Activating Token Index: 968

Text #6

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4295. Min Act: -0.0000

Data Index: 1389695 (The Pile)

Max Activating Token Index: 182

Text #7

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4612. Min Act: -0.0001

Data Index: 556670 (The Pile)

Max Activating Token Index: 545

Text #8

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4376. Min Act: -0.0001

Data Index: 691053 (The Pile)

Max Activating Token Index: 315

Text #9

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4247. Min Act: -0.0001

Data Index: 1741244 (The Pile)

Max Activating Token Index: 964

Text #10

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4368. Min Act: -0.0001

Data Index: 1273134 (The Pile)

Max Activating Token Index: 452

Text #11

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4240. Min Act: -0.0001

Data Index: 1738326 (The Pile)

Max Activating Token Index: 147

Text #12

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4415. Min Act: -0.0001

Data Index: 674858 (The Pile)

Max Activating Token Index: 180

Text #13

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4077. Min Act: -0.0000

Data Index: 532893 (The Pile)

Max Activating Token Index: 395

Text #14

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4050. Min Act: -0.0000

Data Index: 1661541 (The Pile)

Max Activating Token Index: 1009

Text #15

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4196. Min Act: -0.0000

Data Index: 742016 (The Pile)

Max Activating Token Index: 129

Text #16

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4134. Min Act: -0.0000

Data Index: 706569 (The Pile)

Max Activating Token Index: 856

Text #17

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4136. Min Act: -0.0001

Data Index: 570807 (The Pile)

Max Activating Token Index: 411

Text #18

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4252. Min Act: -0.0001

Data Index: 729666 (The Pile)

Max Activating Token Index: 192

Text #19

Max Range: 0.5335. Min Range: -0.5335

Max Act: 0.4142. Min Act: -0.0001

Data Index: 1782718 (The Pile)

Max Activating Token Index: 199