Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2532 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.7661. Min Act: -0.0000

Data Index: 1032555 (The Pile)

Max Activating Token Index: 734

Text #1

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.9343. Min Act: -0.0000

Data Index: 409867 (The Pile)

Max Activating Token Index: 157

Text #2

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.7784. Min Act: 0.0000

Data Index: 479030 (The Pile)

Max Activating Token Index: 416

Text #3

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6623. Min Act: -0.0000

Data Index: 1849935 (The Pile)

Max Activating Token Index: 830

Text #4

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.7573. Min Act: -0.0000

Data Index: 980609 (The Pile)

Max Activating Token Index: 120

Text #5

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.7603. Min Act: -0.0000

Data Index: 571930 (The Pile)

Max Activating Token Index: 251

Text #6

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6384. Min Act: -0.0000

Data Index: 416787 (The Pile)

Max Activating Token Index: 712

Text #7

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6940. Min Act: -0.0000

Data Index: 920616 (The Pile)

Max Activating Token Index: 213

Text #8

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6847. Min Act: -0.0000

Data Index: 1242533 (The Pile)

Max Activating Token Index: 993

Text #9

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6182. Min Act: -0.0000

Data Index: 1717859 (The Pile)

Max Activating Token Index: 774

Text #10

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6598. Min Act: -0.0000

Data Index: 1066187 (The Pile)

Max Activating Token Index: 741

Text #11

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.8369. Min Act: -0.0000

Data Index: 657570 (The Pile)

Max Activating Token Index: 708

Text #12

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.5782. Min Act: -0.0000

Data Index: 933812 (The Pile)

Max Activating Token Index: 753

Text #13

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6175. Min Act: -0.0000

Data Index: 497658 (The Pile)

Max Activating Token Index: 605

Text #14

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6150. Min Act: -0.0000

Data Index: 997781 (The Pile)

Max Activating Token Index: 936

Text #15

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.5827. Min Act: -0.0000

Data Index: 320210 (The Pile)

Max Activating Token Index: 862

Text #16

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6636. Min Act: -0.0000

Data Index: 1430105 (The Pile)

Max Activating Token Index: 732

Text #17

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.5634. Min Act: -0.0000

Data Index: 1121829 (The Pile)

Max Activating Token Index: 761

Text #18

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.6829. Min Act: -0.0000

Data Index: 1879510 (The Pile)

Max Activating Token Index: 860

Text #19

Max Range: 0.9343. Min Range: -0.9343

Max Act: 0.5582. Min Act: -0.0000

Data Index: 1398788 (The Pile)

Max Activating Token Index: 717