Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2941 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.8588. Min Act: -0.0000

Data Index: 1504258 (The Pile)

Max Activating Token Index: 489

Text #1

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.6614. Min Act: -0.0000

Data Index: 642396 (The Pile)

Max Activating Token Index: 508

Text #2

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.6199. Min Act: -0.0000

Data Index: 1185216 (The Pile)

Max Activating Token Index: 313

Text #3

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.6227. Min Act: -0.0000

Data Index: 944186 (The Pile)

Max Activating Token Index: 1009

Text #4

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5521. Min Act: -0.0000

Data Index: 145693 (The Pile)

Max Activating Token Index: 337

Text #5

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5639. Min Act: -0.0000

Data Index: 1849168 (The Pile)

Max Activating Token Index: 626

Text #6

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5144. Min Act: -0.0000

Data Index: 3005 (The Pile)

Max Activating Token Index: 832

Text #7

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5759. Min Act: -0.0000

Data Index: 1238703 (The Pile)

Max Activating Token Index: 758

Text #8

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4725. Min Act: -0.0000

Data Index: 47151 (The Pile)

Max Activating Token Index: 565

Text #9

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5301. Min Act: -0.0000

Data Index: 1259332 (The Pile)

Max Activating Token Index: 118

Text #10

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5827. Min Act: -0.0000

Data Index: 1361296 (The Pile)

Max Activating Token Index: 723

Text #11

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4974. Min Act: -0.0000

Data Index: 421011 (The Pile)

Max Activating Token Index: 853

Text #12

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4986. Min Act: -0.0000

Data Index: 907141 (The Pile)

Max Activating Token Index: 514

Text #13

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4823. Min Act: -0.0000

Data Index: 1632825 (The Pile)

Max Activating Token Index: 641

Text #14

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4884. Min Act: -0.0000

Data Index: 1556307 (The Pile)

Max Activating Token Index: 194

Text #15

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5163. Min Act: -0.0000

Data Index: 1987498 (The Pile)

Max Activating Token Index: 752

Text #16

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5279. Min Act: -0.0000

Data Index: 771296 (The Pile)

Max Activating Token Index: 154

Text #17

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4863. Min Act: -0.0000

Data Index: 1512815 (The Pile)

Max Activating Token Index: 496

Text #18

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.5155. Min Act: -0.0000

Data Index: 1670881 (The Pile)

Max Activating Token Index: 596

Text #19

Max Range: 0.8588. Min Range: -0.8588

Max Act: 0.4661. Min Act: -0.0000

Data Index: 1876933 (The Pile)

Max Activating Token Index: 266