Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2649 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.9338. Min Act: -0.0000

Data Index: 1618038 (The Pile)

Max Activating Token Index: 530

Text #1

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.9538. Min Act: -0.0000

Data Index: 335785 (The Pile)

Max Activating Token Index: 108

Text #2

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8678. Min Act: -0.0000

Data Index: 1727355 (The Pile)

Max Activating Token Index: 550

Text #3

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.9241. Min Act: -0.0000

Data Index: 1231885 (The Pile)

Max Activating Token Index: 58

Text #4

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.9058. Min Act: -0.0000

Data Index: 172295 (The Pile)

Max Activating Token Index: 645

Text #5

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8957. Min Act: -0.0000

Data Index: 1614515 (The Pile)

Max Activating Token Index: 833

Text #6

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8535. Min Act: -0.0000

Data Index: 1046643 (The Pile)

Max Activating Token Index: 183

Text #7

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8234. Min Act: -0.0000

Data Index: 1145259 (The Pile)

Max Activating Token Index: 51

Text #8

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8058. Min Act: -0.0000

Data Index: 1176946 (The Pile)

Max Activating Token Index: 212

Text #9

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.9121. Min Act: -0.0000

Data Index: 1380752 (The Pile)

Max Activating Token Index: 538

Text #10

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8276. Min Act: -0.0000

Data Index: 577024 (The Pile)

Max Activating Token Index: 86

Text #11

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8158. Min Act: -0.0000

Data Index: 1611430 (The Pile)

Max Activating Token Index: 57

Text #12

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8125. Min Act: -0.0000

Data Index: 1411924 (The Pile)

Max Activating Token Index: 384

Text #13

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.7833. Min Act: -0.0000

Data Index: 1872635 (The Pile)

Max Activating Token Index: 58

Text #14

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8586. Min Act: -0.0000

Data Index: 615607 (The Pile)

Max Activating Token Index: 318

Text #15

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8079. Min Act: -0.0000

Data Index: 241901 (The Pile)

Max Activating Token Index: 478

Text #16

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8035. Min Act: -0.0000

Data Index: 513785 (The Pile)

Max Activating Token Index: 342

Text #17

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.8307. Min Act: -0.0000

Data Index: 1133894 (The Pile)

Max Activating Token Index: 538

Text #18

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.7946. Min Act: -0.0000

Data Index: 1903545 (The Pile)

Max Activating Token Index: 270

Text #19

Max Range: 0.9538. Min Range: -0.9538

Max Act: 0.7748. Min Act: -0.0000

Data Index: 450753 (The Pile)

Max Activating Token Index: 514