Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1606 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3806. Min Act: -0.0000

Data Index: 1070982 (The Pile)

Max Activating Token Index: 280

Text #1

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3580. Min Act: -0.0000

Data Index: 897647 (The Pile)

Max Activating Token Index: 822

Text #2

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3450. Min Act: -0.0000

Data Index: 160139 (The Pile)

Max Activating Token Index: 525

Text #3

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3629. Min Act: -0.0000

Data Index: 1028476 (The Pile)

Max Activating Token Index: 160

Text #4

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3377. Min Act: -0.0000

Data Index: 1106003 (The Pile)

Max Activating Token Index: 841

Text #5

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3364. Min Act: -0.0000

Data Index: 1522892 (The Pile)

Max Activating Token Index: 220

Text #6

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3169. Min Act: -0.0000

Data Index: 79267 (The Pile)

Max Activating Token Index: 735

Text #7

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3484. Min Act: -0.0000

Data Index: 625769 (The Pile)

Max Activating Token Index: 228

Text #8

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3213. Min Act: -0.0000

Data Index: 1633228 (The Pile)

Max Activating Token Index: 868

Text #9

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3123. Min Act: -0.0000

Data Index: 1955966 (The Pile)

Max Activating Token Index: 780

Text #10

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.2954. Min Act: -0.0000

Data Index: 117941 (The Pile)

Max Activating Token Index: 1006

Text #11

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3117. Min Act: -0.0000

Data Index: 1708681 (The Pile)

Max Activating Token Index: 473

Text #12

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3104. Min Act: -0.0000

Data Index: 1589550 (The Pile)

Max Activating Token Index: 721

Text #13

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.2757. Min Act: -0.0000

Data Index: 346150 (The Pile)

Max Activating Token Index: 814

Text #14

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3069. Min Act: -0.0000

Data Index: 850361 (The Pile)

Max Activating Token Index: 331

Text #15

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3065. Min Act: -0.0000

Data Index: 1935903 (The Pile)

Max Activating Token Index: 13

Text #16

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.2957. Min Act: -0.0000

Data Index: 1880064 (The Pile)

Max Activating Token Index: 79

Text #17

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3159. Min Act: -0.0000

Data Index: 1796115 (The Pile)

Max Activating Token Index: 215

Text #18

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3147. Min Act: -0.0000

Data Index: 1940964 (The Pile)

Max Activating Token Index: 450

Text #19

Max Range: 0.3806. Min Range: -0.3806

Max Act: 0.3112. Min Act: -0.0000

Data Index: 392145 (The Pile)

Max Activating Token Index: 531