Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3310 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2721. Min Act: -0.0000

Data Index: 29915 (The Pile)

Max Activating Token Index: 346

Text #1

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2700. Min Act: -0.0000

Data Index: 94002 (The Pile)

Max Activating Token Index: 103

Text #2

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2630. Min Act: -0.0000

Data Index: 841419 (The Pile)

Max Activating Token Index: 211

Text #3

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2742. Min Act: -0.0000

Data Index: 757628 (The Pile)

Max Activating Token Index: 368

Text #4

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2626. Min Act: -0.0000

Data Index: 1652979 (The Pile)

Max Activating Token Index: 729

Text #5

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2622. Min Act: -0.0000

Data Index: 837163 (The Pile)

Max Activating Token Index: 1010

Text #6

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2539. Min Act: -0.0000

Data Index: 1661733 (The Pile)

Max Activating Token Index: 150

Text #7

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2406. Min Act: -0.0000

Data Index: 365619 (The Pile)

Max Activating Token Index: 148

Text #8

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2557. Min Act: -0.0000

Data Index: 1187359 (The Pile)

Max Activating Token Index: 793

Text #9

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2362. Min Act: -0.0000

Data Index: 1153175 (The Pile)

Max Activating Token Index: 818

Text #10

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2342. Min Act: -0.0000

Data Index: 168458 (The Pile)

Max Activating Token Index: 39

Text #11

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2349. Min Act: -0.0000

Data Index: 1820910 (The Pile)

Max Activating Token Index: 932

Text #12

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2510. Min Act: -0.0000

Data Index: 1287894 (The Pile)

Max Activating Token Index: 658

Text #13

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2193. Min Act: -0.0000

Data Index: 1423915 (The Pile)

Max Activating Token Index: 368

Text #14

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2157. Min Act: -0.0000

Data Index: 1174904 (The Pile)

Max Activating Token Index: 143

Text #15

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2285. Min Act: -0.0000

Data Index: 1102754 (The Pile)

Max Activating Token Index: 398

Text #16

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2276. Min Act: -0.0000

Data Index: 1793767 (The Pile)

Max Activating Token Index: 201

Text #17

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2194. Min Act: -0.0000

Data Index: 618685 (The Pile)

Max Activating Token Index: 474

Text #18

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2406. Min Act: -0.0000

Data Index: 1368089 (The Pile)

Max Activating Token Index: 382

Text #19

Max Range: 0.2742. Min Range: -0.2742

Max Act: 0.2175. Min Act: -0.0000

Data Index: 1289539 (The Pile)

Max Activating Token Index: 1002