Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2241 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 2.1976. Min Range: -2.1976

Max Act: 2.1915. Min Act: -0.0000

Data Index: 1195411 (The Pile)

Max Activating Token Index: 677

Text #1

Max Range: 2.1976. Min Range: -2.1976

Max Act: 2.1464. Min Act: -0.0000

Data Index: 1596120 (The Pile)

Max Activating Token Index: 514

Text #2

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.9902. Min Act: -0.0000

Data Index: 483602 (The Pile)

Max Activating Token Index: 567

Text #3

Max Range: 2.1976. Min Range: -2.1976

Max Act: 2.1976. Min Act: -0.0000

Data Index: 1471936 (The Pile)

Max Activating Token Index: 617

Text #4

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.9972. Min Act: -0.0000

Data Index: 374918 (The Pile)

Max Activating Token Index: 804

Text #5

Max Range: 2.1976. Min Range: -2.1976

Max Act: 2.0085. Min Act: -0.0000

Data Index: 1116300 (The Pile)

Max Activating Token Index: 550

Text #6

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.9885. Min Act: -0.0000

Data Index: 504989 (The Pile)

Max Activating Token Index: 797

Text #7

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8583. Min Act: -0.0000

Data Index: 651691 (The Pile)

Max Activating Token Index: 958

Text #8

Max Range: 2.1976. Min Range: -2.1976

Max Act: 2.0427. Min Act: -0.0000

Data Index: 106166 (The Pile)

Max Activating Token Index: 650

Text #9

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8414. Min Act: -0.0000

Data Index: 140143 (The Pile)

Max Activating Token Index: 549

Text #10

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8239. Min Act: -0.0000

Data Index: 47073 (The Pile)

Max Activating Token Index: 105

Text #11

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8907. Min Act: -0.0000

Data Index: 905752 (The Pile)

Max Activating Token Index: 496

Text #12

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8996. Min Act: -0.0000

Data Index: 693226 (The Pile)

Max Activating Token Index: 148

Text #13

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8782. Min Act: -0.0000

Data Index: 543267 (The Pile)

Max Activating Token Index: 761

Text #14

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8259. Min Act: -0.0000

Data Index: 88178 (The Pile)

Max Activating Token Index: 479

Text #15

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8914. Min Act: -0.0000

Data Index: 560415 (The Pile)

Max Activating Token Index: 333

Text #16

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.9107. Min Act: -0.0000

Data Index: 1577033 (The Pile)

Max Activating Token Index: 348

Text #17

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.7129. Min Act: -0.0000

Data Index: 288182 (The Pile)

Max Activating Token Index: 971

Text #18

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.7800. Min Act: -0.0000

Data Index: 1564447 (The Pile)

Max Activating Token Index: 1020

Text #19

Max Range: 2.1976. Min Range: -2.1976

Max Act: 1.8612. Min Act: -0.0000

Data Index: 139872 (The Pile)

Max Activating Token Index: 684