Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1093 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.0358. Min Range: -3.0358

Max Act: 3.0358. Min Act: -0.0000

Data Index: 648359 (The Pile)

Max Activating Token Index: 396

Text #1

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.7542. Min Act: -0.0000

Data Index: 1175842 (The Pile)

Max Activating Token Index: 142

Text #2

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.8766. Min Act: -0.0000

Data Index: 1562225 (The Pile)

Max Activating Token Index: 368

Text #3

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.7961. Min Act: -0.0000

Data Index: 703528 (The Pile)

Max Activating Token Index: 279

Text #4

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.7436. Min Act: -0.0000

Data Index: 1256943 (The Pile)

Max Activating Token Index: 858

Text #5

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4506. Min Act: -0.0000

Data Index: 283175 (The Pile)

Max Activating Token Index: 788

Text #6

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.3415. Min Act: -0.0000

Data Index: 1407794 (The Pile)

Max Activating Token Index: 1001

Text #7

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.7018. Min Act: -0.0000

Data Index: 1928009 (The Pile)

Max Activating Token Index: 713

Text #8

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.6848. Min Act: -0.0000

Data Index: 1739060 (The Pile)

Max Activating Token Index: 155

Text #9

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.6523. Min Act: -0.0000

Data Index: 290824 (The Pile)

Max Activating Token Index: 481

Text #10

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.7423. Min Act: -0.0000

Data Index: 1120094 (The Pile)

Max Activating Token Index: 831

Text #11

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.6131. Min Act: -0.0000

Data Index: 1036608 (The Pile)

Max Activating Token Index: 273

Text #12

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.5806. Min Act: -0.0000

Data Index: 56190 (The Pile)

Max Activating Token Index: 108

Text #13

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4226. Min Act: -0.0000

Data Index: 1130234 (The Pile)

Max Activating Token Index: 880

Text #14

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.5415. Min Act: -0.0000

Data Index: 1930653 (The Pile)

Max Activating Token Index: 238

Text #15

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4897. Min Act: -0.0000

Data Index: 1850628 (The Pile)

Max Activating Token Index: 652

Text #16

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4861. Min Act: -0.0000

Data Index: 1719591 (The Pile)

Max Activating Token Index: 327

Text #17

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.2325. Min Act: -0.0000

Data Index: 818783 (The Pile)

Max Activating Token Index: 820

Text #18

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4661. Min Act: -0.0000

Data Index: 127915 (The Pile)

Max Activating Token Index: 949

Text #19

Max Range: 3.0358. Min Range: -3.0358

Max Act: 2.4726. Min Act: -0.0000

Data Index: 722848 (The Pile)

Max Activating Token Index: 681