Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2430 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.4625. Min Act: -0.0000

Data Index: 1181782 (The Pile)

Max Activating Token Index: 969

Text #1

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3853. Min Act: -0.0000

Data Index: 819008 (The Pile)

Max Activating Token Index: 978

Text #2

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3347. Min Act: -0.0000

Data Index: 1925506 (The Pile)

Max Activating Token Index: 930

Text #3

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3790. Min Act: -0.0000

Data Index: 258269 (The Pile)

Max Activating Token Index: 824

Text #4

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3308. Min Act: -0.0000

Data Index: 1018316 (The Pile)

Max Activating Token Index: 770

Text #5

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3893. Min Act: -0.0000

Data Index: 911703 (The Pile)

Max Activating Token Index: 686

Text #6

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3514. Min Act: -0.0000

Data Index: 1194376 (The Pile)

Max Activating Token Index: 1018

Text #7

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3399. Min Act: -0.0000

Data Index: 1615372 (The Pile)

Max Activating Token Index: 933

Text #8

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3143. Min Act: -0.0000

Data Index: 988375 (The Pile)

Max Activating Token Index: 667

Text #9

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3476. Min Act: -0.0000

Data Index: 886611 (The Pile)

Max Activating Token Index: 288

Text #10

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3325. Min Act: -0.0000

Data Index: 339762 (The Pile)

Max Activating Token Index: 561

Text #11

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2744. Min Act: -0.0000

Data Index: 1696996 (The Pile)

Max Activating Token Index: 800

Text #12

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3046. Min Act: -0.0000

Data Index: 933201 (The Pile)

Max Activating Token Index: 153

Text #13

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2875. Min Act: -0.0000

Data Index: 2788 (The Pile)

Max Activating Token Index: 227

Text #14

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2756. Min Act: -0.0000

Data Index: 503975 (The Pile)

Max Activating Token Index: 682

Text #15

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2299. Min Act: -0.0000

Data Index: 1122769 (The Pile)

Max Activating Token Index: 758

Text #16

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2728. Min Act: -0.0000

Data Index: 190442 (The Pile)

Max Activating Token Index: 628

Text #17

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.2654. Min Act: -0.0000

Data Index: 917239 (The Pile)

Max Activating Token Index: 603

Text #18

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3012. Min Act: -0.0000

Data Index: 326367 (The Pile)

Max Activating Token Index: 998

Text #19

Max Range: 0.4625. Min Range: -0.4625

Max Act: 0.3302. Min Act: -0.0000

Data Index: 1772139 (The Pile)

Max Activating Token Index: 661