Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1307 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 8.7272. Min Range: -8.7272

Max Act: 8.0242. Min Act: -0.0000

Data Index: 1483159 (The Pile)

Max Activating Token Index: 586

Text #1

Max Range: 8.7272. Min Range: -8.7272

Max Act: 8.7272. Min Act: -0.0000

Data Index: 671461 (The Pile)

Max Activating Token Index: 144

Text #2

Max Range: 8.7272. Min Range: -8.7272

Max Act: 8.0468. Min Act: -0.0000

Data Index: 117207 (The Pile)

Max Activating Token Index: 747

Text #3

Max Range: 8.7272. Min Range: -8.7272

Max Act: 8.4200. Min Act: -0.0000

Data Index: 1619811 (The Pile)

Max Activating Token Index: 112

Text #4

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.9991. Min Act: -0.0000

Data Index: 768395 (The Pile)

Max Activating Token Index: 461

Text #5

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.9143. Min Act: -0.0000

Data Index: 792497 (The Pile)

Max Activating Token Index: 638

Text #6

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.7598. Min Act: -0.0000

Data Index: 1396185 (The Pile)

Max Activating Token Index: 741

Text #7

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.3963. Min Act: -0.0000

Data Index: 915966 (The Pile)

Max Activating Token Index: 337

Text #8

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.7382. Min Act: -0.0000

Data Index: 1711214 (The Pile)

Max Activating Token Index: 435

Text #9

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.5319. Min Act: -0.0000

Data Index: 725848 (The Pile)

Max Activating Token Index: 34

Text #10

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.0965. Min Act: -0.0000

Data Index: 414520 (The Pile)

Max Activating Token Index: 578

Text #11

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.9890. Min Act: -0.0000

Data Index: 944765 (The Pile)

Max Activating Token Index: 509

Text #12

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.2258. Min Act: -0.0000

Data Index: 1270227 (The Pile)

Max Activating Token Index: 327

Text #13

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.3103. Min Act: -0.0000

Data Index: 498914 (The Pile)

Max Activating Token Index: 588

Text #14

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.3387. Min Act: -0.0000

Data Index: 1579750 (The Pile)

Max Activating Token Index: 914

Text #15

Max Range: 8.7272. Min Range: -8.7272

Max Act: 7.0529. Min Act: -0.0000

Data Index: 767070 (The Pile)

Max Activating Token Index: 1002

Text #16

Max Range: 8.7272. Min Range: -8.7272

Max Act: 5.4970. Min Act: -0.0000

Data Index: 1175860 (The Pile)

Max Activating Token Index: 745

Text #17

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.7918. Min Act: -0.0000

Data Index: 777784 (The Pile)

Max Activating Token Index: 678

Text #18

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.4965. Min Act: -0.0000

Data Index: 797733 (The Pile)

Max Activating Token Index: 959

Text #19

Max Range: 8.7272. Min Range: -8.7272

Max Act: 6.3459. Min Act: -0.0000

Data Index: 843326 (The Pile)

Max Activating Token Index: 995