Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 666 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.1993. Min Act: -0.0000

Data Index: 997038 (The Pile)

Max Activating Token Index: 738

Text #1

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.1834. Min Act: -0.0000

Data Index: 126949 (The Pile)

Max Activating Token Index: 691

Text #2

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.1026. Min Act: -0.0000

Data Index: 1669563 (The Pile)

Max Activating Token Index: 820

Text #3

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.1338. Min Act: -0.0000

Data Index: 1704195 (The Pile)

Max Activating Token Index: 58

Text #4

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.1000. Min Act: -0.0000

Data Index: 1403914 (The Pile)

Max Activating Token Index: 319

Text #5

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0572. Min Act: -0.0000

Data Index: 1047993 (The Pile)

Max Activating Token Index: 287

Text #6

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0851. Min Act: -0.0000

Data Index: 960181 (The Pile)

Max Activating Token Index: 685

Text #7

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0270. Min Act: -0.0000

Data Index: 255972 (The Pile)

Max Activating Token Index: 116

Text #8

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0362. Min Act: -0.0000

Data Index: 1448961 (The Pile)

Max Activating Token Index: 300

Text #9

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0416. Min Act: -0.0000

Data Index: 280518 (The Pile)

Max Activating Token Index: 848

Text #10

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0077. Min Act: -0.0000

Data Index: 1642118 (The Pile)

Max Activating Token Index: 378

Text #11

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9757. Min Act: -0.0000

Data Index: 1765647 (The Pile)

Max Activating Token Index: 354

Text #12

Max Range: 1.1993. Min Range: -1.1993

Max Act: 1.0413. Min Act: -0.0000

Data Index: 1505514 (The Pile)

Max Activating Token Index: 691

Text #13

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9478. Min Act: -0.0000

Data Index: 607727 (The Pile)

Max Activating Token Index: 1001

Text #14

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9874. Min Act: -0.0000

Data Index: 911166 (The Pile)

Max Activating Token Index: 325

Text #15

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9883. Min Act: -0.0000

Data Index: 1934076 (The Pile)

Max Activating Token Index: 478

Text #16

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9568. Min Act: -0.0000

Data Index: 718445 (The Pile)

Max Activating Token Index: 571

Text #17

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9725. Min Act: -0.0000

Data Index: 1949912 (The Pile)

Max Activating Token Index: 918

Text #18

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9330. Min Act: -0.0000

Data Index: 1355312 (The Pile)

Max Activating Token Index: 425

Text #19

Max Range: 1.1993. Min Range: -1.1993

Max Act: 0.9055. Min Act: -0.0000

Data Index: 1187079 (The Pile)

Max Activating Token Index: 769