Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1605 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1004. Min Range: -1.1004

Max Act: 1.0295. Min Act: -0.0000

Data Index: 400781 (The Pile)

Max Activating Token Index: 396

Text #1

Max Range: 1.1004. Min Range: -1.1004

Max Act: 1.0894. Min Act: -0.0000

Data Index: 1762262 (The Pile)

Max Activating Token Index: 721

Text #2

Max Range: 1.1004. Min Range: -1.1004

Max Act: 1.1004. Min Act: -0.0000

Data Index: 412133 (The Pile)

Max Activating Token Index: 409

Text #3

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.9087. Min Act: -0.0000

Data Index: 1225934 (The Pile)

Max Activating Token Index: 842

Text #4

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.7373. Min Act: -0.0000

Data Index: 874107 (The Pile)

Max Activating Token Index: 103

Text #5

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.8983. Min Act: -0.0000

Data Index: 808811 (The Pile)

Max Activating Token Index: 522

Text #6

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.7705. Min Act: -0.0000

Data Index: 1116045 (The Pile)

Max Activating Token Index: 987

Text #7

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.7136. Min Act: -0.0000

Data Index: 1585230 (The Pile)

Max Activating Token Index: 840

Text #8

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.8232. Min Act: -0.0000

Data Index: 10315 (The Pile)

Max Activating Token Index: 210

Text #9

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6414. Min Act: -0.0000

Data Index: 151921 (The Pile)

Max Activating Token Index: 940

Text #10

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.7261. Min Act: -0.0000

Data Index: 1568237 (The Pile)

Max Activating Token Index: 167

Text #11

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.7192. Min Act: -0.0000

Data Index: 737772 (The Pile)

Max Activating Token Index: 923

Text #12

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6672. Min Act: -0.0000

Data Index: 243153 (The Pile)

Max Activating Token Index: 590

Text #13

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6094. Min Act: -0.0000

Data Index: 1701340 (The Pile)

Max Activating Token Index: 621

Text #14

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6407. Min Act: -0.0000

Data Index: 1419620 (The Pile)

Max Activating Token Index: 395

Text #15

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6854. Min Act: -0.0000

Data Index: 1608885 (The Pile)

Max Activating Token Index: 624

Text #16

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6332. Min Act: -0.0000

Data Index: 429891 (The Pile)

Max Activating Token Index: 482

Text #17

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6971. Min Act: -0.0000

Data Index: 1248949 (The Pile)

Max Activating Token Index: 700

Text #18

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.5679. Min Act: -0.0000

Data Index: 1229590 (The Pile)

Max Activating Token Index: 324

Text #19

Max Range: 1.1004. Min Range: -1.1004

Max Act: 0.6136. Min Act: -0.0000

Data Index: 1524831 (The Pile)

Max Activating Token Index: 334