Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1477 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.9209. Min Act: -0.0000

Data Index: 609380 (The Pile)

Max Activating Token Index: 570

Text #1

Max Range: 3.0639. Min Range: -3.0639

Max Act: 3.0639. Min Act: -0.0000

Data Index: 862836 (The Pile)

Max Activating Token Index: 997

Text #2

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.7787. Min Act: -0.0000

Data Index: 1813930 (The Pile)

Max Activating Token Index: 783

Text #3

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.4801. Min Act: -0.0000

Data Index: 317922 (The Pile)

Max Activating Token Index: 296

Text #4

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.5148. Min Act: -0.0000

Data Index: 748114 (The Pile)

Max Activating Token Index: 73

Text #5

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.3770. Min Act: -0.0000

Data Index: 1820451 (The Pile)

Max Activating Token Index: 174

Text #6

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.4277. Min Act: -0.0000

Data Index: 1918250 (The Pile)

Max Activating Token Index: 780

Text #7

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.1448. Min Act: -0.0000

Data Index: 130142 (The Pile)

Max Activating Token Index: 314

Text #8

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.2573. Min Act: -0.0000

Data Index: 773942 (The Pile)

Max Activating Token Index: 634

Text #9

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.2316. Min Act: -0.0000

Data Index: 553231 (The Pile)

Max Activating Token Index: 710

Text #10

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.1585. Min Act: -0.0000

Data Index: 1006846 (The Pile)

Max Activating Token Index: 288

Text #11

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.2191. Min Act: -0.0000

Data Index: 772038 (The Pile)

Max Activating Token Index: 763

Text #12

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.2133. Min Act: -0.0000

Data Index: 1570380 (The Pile)

Max Activating Token Index: 105

Text #13

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.0832. Min Act: 0.0000

Data Index: 513882 (The Pile)

Max Activating Token Index: 418

Text #14

Max Range: 3.0639. Min Range: -3.0639

Max Act: 1.9770. Min Act: -0.0000

Data Index: 1111883 (The Pile)

Max Activating Token Index: 813

Text #15

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.0722. Min Act: -0.0000

Data Index: 745012 (The Pile)

Max Activating Token Index: 365

Text #16

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.0957. Min Act: -0.0000

Data Index: 1533312 (The Pile)

Max Activating Token Index: 352

Text #17

Max Range: 3.0639. Min Range: -3.0639

Max Act: 1.9733. Min Act: -0.0000

Data Index: 1338120 (The Pile)

Max Activating Token Index: 902

Text #18

Max Range: 3.0639. Min Range: -3.0639

Max Act: 1.9541. Min Act: -0.0000

Data Index: 369328 (The Pile)

Max Activating Token Index: 140

Text #19

Max Range: 3.0639. Min Range: -3.0639

Max Act: 2.0728. Min Act: -0.0000

Data Index: 676226 (The Pile)

Max Activating Token Index: 203