Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4279 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.6154. Min Act: -0.0000

Data Index: 539960 (The Pile)

Max Activating Token Index: 397

Text #1

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.5739. Min Act: -0.0000

Data Index: 1702947 (The Pile)

Max Activating Token Index: 76

Text #2

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.5563. Min Act: 0.0000

Data Index: 415584 (The Pile)

Max Activating Token Index: 185

Text #3

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.5575. Min Act: -0.0000

Data Index: 232975 (The Pile)

Max Activating Token Index: 533

Text #4

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4677. Min Act: -0.0000

Data Index: 416479 (The Pile)

Max Activating Token Index: 558

Text #5

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4994. Min Act: -0.0000

Data Index: 1234512 (The Pile)

Max Activating Token Index: 560

Text #6

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4383. Min Act: -0.0000

Data Index: 1743361 (The Pile)

Max Activating Token Index: 514

Text #7

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4352. Min Act: -0.0000

Data Index: 643894 (The Pile)

Max Activating Token Index: 831

Text #8

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4515. Min Act: -0.0000

Data Index: 1315667 (The Pile)

Max Activating Token Index: 539

Text #9

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4200. Min Act: -0.0000

Data Index: 1188018 (The Pile)

Max Activating Token Index: 263

Text #10

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.4463. Min Act: -0.0000

Data Index: 1501756 (The Pile)

Max Activating Token Index: 991

Text #11

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3640. Min Act: -0.0000

Data Index: 180715 (The Pile)

Max Activating Token Index: 910

Text #12

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3810. Min Act: -0.0000

Data Index: 1136224 (The Pile)

Max Activating Token Index: 212

Text #13

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3865. Min Act: -0.0000

Data Index: 1218859 (The Pile)

Max Activating Token Index: 436

Text #14

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3928. Min Act: -0.0000

Data Index: 788650 (The Pile)

Max Activating Token Index: 127

Text #15

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3823. Min Act: -0.0000

Data Index: 1543614 (The Pile)

Max Activating Token Index: 296

Text #16

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3701. Min Act: -0.0000

Data Index: 476477 (The Pile)

Max Activating Token Index: 664

Text #17

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3720. Min Act: -0.0000

Data Index: 78733 (The Pile)

Max Activating Token Index: 500

Text #18

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3340. Min Act: -0.0000

Data Index: 222713 (The Pile)

Max Activating Token Index: 959

Text #19

Max Range: 0.6154. Min Range: -0.6154

Max Act: 0.3670. Min Act: -0.0000

Data Index: 240611 (The Pile)

Max Activating Token Index: 184