Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3009 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1401. Min Range: -3.1401

Max Act: 3.1401. Min Act: -0.0000

Data Index: 1862937 (The Pile)

Max Activating Token Index: 756

Text #1

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.8914. Min Act: -0.0000

Data Index: 705899 (The Pile)

Max Activating Token Index: 587

Text #2

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.7752. Min Act: -0.0000

Data Index: 1193030 (The Pile)

Max Activating Token Index: 32

Text #3

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.7466. Min Act: 0.0000

Data Index: 1927672 (The Pile)

Max Activating Token Index: 81

Text #4

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.6295. Min Act: -0.0000

Data Index: 1745490 (The Pile)

Max Activating Token Index: 60

Text #5

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.7426. Min Act: -0.0000

Data Index: 872962 (The Pile)

Max Activating Token Index: 218

Text #6

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.3246. Min Act: -0.0000

Data Index: 1226832 (The Pile)

Max Activating Token Index: 127

Text #7

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.4265. Min Act: -0.0000

Data Index: 745796 (The Pile)

Max Activating Token Index: 638

Text #8

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.5012. Min Act: -0.0000

Data Index: 433681 (The Pile)

Max Activating Token Index: 519

Text #9

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.5701. Min Act: -0.0000

Data Index: 864030 (The Pile)

Max Activating Token Index: 1009

Text #10

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.4400. Min Act: -0.0000

Data Index: 315420 (The Pile)

Max Activating Token Index: 560

Text #11

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.3142. Min Act: -0.0000

Data Index: 916152 (The Pile)

Max Activating Token Index: 32

Text #12

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.3175. Min Act: -0.0000

Data Index: 1169486 (The Pile)

Max Activating Token Index: 417

Text #13

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.4178. Min Act: -0.0000

Data Index: 270929 (The Pile)

Max Activating Token Index: 104

Text #14

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.2920. Min Act: -0.0000

Data Index: 931714 (The Pile)

Max Activating Token Index: 999

Text #15

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.2831. Min Act: -0.0000

Data Index: 1000326 (The Pile)

Max Activating Token Index: 566

Text #16

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.4011. Min Act: -0.0000

Data Index: 1398549 (The Pile)

Max Activating Token Index: 494

Text #17

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.2349. Min Act: -0.0000

Data Index: 539993 (The Pile)

Max Activating Token Index: 818

Text #18

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.2418. Min Act: -0.0000

Data Index: 730720 (The Pile)

Max Activating Token Index: 634

Text #19

Max Range: 3.1401. Min Range: -3.1401

Max Act: 2.3462. Min Act: -0.0000

Data Index: 482439 (The Pile)

Max Activating Token Index: 342