Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2385 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1197. Min Range: -1.1197

Max Act: 1.1197. Min Act: -0.0000

Data Index: 256455 (The Pile)

Max Activating Token Index: 768

Text #1

Max Range: 1.1197. Min Range: -1.1197

Max Act: 1.0557. Min Act: -0.0000

Data Index: 570884 (The Pile)

Max Activating Token Index: 473

Text #2

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.9052. Min Act: -0.0000

Data Index: 88567 (The Pile)

Max Activating Token Index: 312

Text #3

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7636. Min Act: -0.0000

Data Index: 1656402 (The Pile)

Max Activating Token Index: 220

Text #4

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8759. Min Act: -0.0000

Data Index: 1230314 (The Pile)

Max Activating Token Index: 161

Text #5

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8160. Min Act: -0.0000

Data Index: 1213842 (The Pile)

Max Activating Token Index: 974

Text #6

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.0770. Min Act: -0.0000

Data Index: 1910439 (The Pile)

Max Activating Token Index: 707

Text #7

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8339. Min Act: -0.0000

Data Index: 1868028 (The Pile)

Max Activating Token Index: 361

Text #8

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8726. Min Act: -0.0000

Data Index: 636704 (The Pile)

Max Activating Token Index: 486

Text #9

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7564. Min Act: -0.0000

Data Index: 1731943 (The Pile)

Max Activating Token Index: 832

Text #10

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8019. Min Act: -0.0000

Data Index: 1548351 (The Pile)

Max Activating Token Index: 940

Text #11

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7978. Min Act: -0.0000

Data Index: 640569 (The Pile)

Max Activating Token Index: 959

Text #12

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.8101. Min Act: -0.0000

Data Index: 629521 (The Pile)

Max Activating Token Index: 785

Text #13

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7334. Min Act: -0.0000

Data Index: 1414303 (The Pile)

Max Activating Token Index: 387

Text #14

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7411. Min Act: -0.0000

Data Index: 1980698 (The Pile)

Max Activating Token Index: 471

Text #15

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7655. Min Act: -0.0000

Data Index: 1180290 (The Pile)

Max Activating Token Index: 461

Text #16

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7032. Min Act: -0.0000

Data Index: 499053 (The Pile)

Max Activating Token Index: 271

Text #17

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7111. Min Act: -0.0000

Data Index: 992439 (The Pile)

Max Activating Token Index: 831

Text #18

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7139. Min Act: -0.0000

Data Index: 1984978 (The Pile)

Max Activating Token Index: 23

Text #19

Max Range: 1.1197. Min Range: -1.1197

Max Act: 0.7541. Min Act: -0.0000

Data Index: 305821 (The Pile)

Max Activating Token Index: 540