Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1515 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.7350. Min Act: -0.0000

Data Index: 100446 (The Pile)

Max Activating Token Index: 13

Text #1

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.5337. Min Act: -0.0000

Data Index: 1727322 (The Pile)

Max Activating Token Index: 13

Text #2

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.4312. Min Act: -0.0000

Data Index: 943915 (The Pile)

Max Activating Token Index: 8

Text #3

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.4312. Min Act: -0.0000

Data Index: 666135 (The Pile)

Max Activating Token Index: 8

Text #4

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.4312. Min Act: -0.0000

Data Index: 644999 (The Pile)

Max Activating Token Index: 8

Text #5

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2155. Min Act: -0.0000

Data Index: 1513012 (The Pile)

Max Activating Token Index: 994

Text #6

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.5837. Min Act: -0.0000

Data Index: 182418 (The Pile)

Max Activating Token Index: 831

Text #7

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.3054. Min Act: -0.0000

Data Index: 1364757 (The Pile)

Max Activating Token Index: 13

Text #8

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.3032. Min Act: -0.0000

Data Index: 1867223 (The Pile)

Max Activating Token Index: 13

Text #9

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.4822. Min Act: -0.0000

Data Index: 386332 (The Pile)

Max Activating Token Index: 12

Text #10

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2466. Min Act: -0.0000

Data Index: 758203 (The Pile)

Max Activating Token Index: 499

Text #11

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2605. Min Act: -0.0000

Data Index: 399837 (The Pile)

Max Activating Token Index: 7

Text #12

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2605. Min Act: -0.0000

Data Index: 771566 (The Pile)

Max Activating Token Index: 7

Text #13

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.4441. Min Act: -0.0000

Data Index: 1120496 (The Pile)

Max Activating Token Index: 11

Text #14

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2488. Min Act: -0.0000

Data Index: 802722 (The Pile)

Max Activating Token Index: 10

Text #15

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.3051. Min Act: -0.0000

Data Index: 302751 (The Pile)

Max Activating Token Index: 12

Text #16

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2827. Min Act: -0.0000

Data Index: 690349 (The Pile)

Max Activating Token Index: 12

Text #17

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2759. Min Act: -0.0000

Data Index: 1054489 (The Pile)

Max Activating Token Index: 10

Text #18

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2446. Min Act: -0.0000

Data Index: 384328 (The Pile)

Max Activating Token Index: 10

Text #19

Max Range: 5.7350. Min Range: -5.7350

Max Act: 5.2168. Min Act: -0.0000

Data Index: 994608 (The Pile)

Max Activating Token Index: 6