Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 385 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2873. Min Act: -0.0000

Data Index: 207327 (The Pile)

Max Activating Token Index: 183

Text #1

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2955. Min Act: -0.0000

Data Index: 1515219 (The Pile)

Max Activating Token Index: 927

Text #2

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2826. Min Act: -0.0000

Data Index: 1363335 (The Pile)

Max Activating Token Index: 253

Text #3

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2682. Min Act: -0.0000

Data Index: 1006723 (The Pile)

Max Activating Token Index: 615

Text #4

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2629. Min Act: -0.0000

Data Index: 1359939 (The Pile)

Max Activating Token Index: 956

Text #5

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2746. Min Act: -0.0000

Data Index: 657284 (The Pile)

Max Activating Token Index: 1018

Text #6

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2566. Min Act: -0.0000

Data Index: 569856 (The Pile)

Max Activating Token Index: 653

Text #7

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2610. Min Act: -0.0000

Data Index: 566262 (The Pile)

Max Activating Token Index: 1015

Text #8

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2630. Min Act: -0.0000

Data Index: 711208 (The Pile)

Max Activating Token Index: 996

Text #9

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2455. Min Act: -0.0000

Data Index: 546648 (The Pile)

Max Activating Token Index: 267

Text #10

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2283. Min Act: 0.0000

Data Index: 896774 (The Pile)

Max Activating Token Index: 875

Text #11

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2128. Min Act: -0.0000

Data Index: 1316393 (The Pile)

Max Activating Token Index: 619

Text #12

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2287. Min Act: -0.0000

Data Index: 1862936 (The Pile)

Max Activating Token Index: 714

Text #13

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2225. Min Act: 0.0000

Data Index: 263203 (The Pile)

Max Activating Token Index: 893

Text #14

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2436. Min Act: -0.0000

Data Index: 559220 (The Pile)

Max Activating Token Index: 339

Text #15

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2282. Min Act: 0.0000

Data Index: 1249775 (The Pile)

Max Activating Token Index: 386

Text #16

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2492. Min Act: -0.0000

Data Index: 1697394 (The Pile)

Max Activating Token Index: 621

Text #17

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2316. Min Act: -0.0000

Data Index: 1066735 (The Pile)

Max Activating Token Index: 747

Text #18

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2420. Min Act: -0.0000

Data Index: 1572681 (The Pile)

Max Activating Token Index: 937

Text #19

Max Range: 0.2955. Min Range: -0.2955

Max Act: 0.2535. Min Act: -0.0000

Data Index: 662487 (The Pile)

Max Activating Token Index: 654