Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2619 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1854. Min Range: -1.1854

Max Act: 1.1854. Min Act: -0.0000

Data Index: 1317116 (The Pile)

Max Activating Token Index: 431

Text #1

Max Range: 1.1854. Min Range: -1.1854

Max Act: 1.1232. Min Act: -0.0000

Data Index: 472942 (The Pile)

Max Activating Token Index: 780

Text #2

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.9169. Min Act: -0.0000

Data Index: 1218295 (The Pile)

Max Activating Token Index: 531

Text #3

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.9352. Min Act: -0.0000

Data Index: 263101 (The Pile)

Max Activating Token Index: 512

Text #4

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.8491. Min Act: -0.0000

Data Index: 1474070 (The Pile)

Max Activating Token Index: 616

Text #5

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.9519. Min Act: -0.0000

Data Index: 1985168 (The Pile)

Max Activating Token Index: 97

Text #6

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7401. Min Act: -0.0000

Data Index: 909601 (The Pile)

Max Activating Token Index: 444

Text #7

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.5510. Min Act: -0.0000

Data Index: 1502192 (The Pile)

Max Activating Token Index: 781

Text #8

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.8620. Min Act: -0.0000

Data Index: 442283 (The Pile)

Max Activating Token Index: 669

Text #9

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7621. Min Act: -0.0000

Data Index: 413063 (The Pile)

Max Activating Token Index: 659

Text #10

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.8920. Min Act: -0.0000

Data Index: 1061261 (The Pile)

Max Activating Token Index: 60

Text #11

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.9432. Min Act: -0.0000

Data Index: 972980 (The Pile)

Max Activating Token Index: 826

Text #12

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.8738. Min Act: -0.0000

Data Index: 1317441 (The Pile)

Max Activating Token Index: 762

Text #13

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7592. Min Act: -0.0000

Data Index: 335069 (The Pile)

Max Activating Token Index: 552

Text #14

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7511. Min Act: -0.0000

Data Index: 7384 (The Pile)

Max Activating Token Index: 80

Text #15

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7572. Min Act: -0.0000

Data Index: 1173036 (The Pile)

Max Activating Token Index: 579

Text #16

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.5272. Min Act: -0.0000

Data Index: 1530403 (The Pile)

Max Activating Token Index: 878

Text #17

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7688. Min Act: -0.0000

Data Index: 377831 (The Pile)

Max Activating Token Index: 459

Text #18

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7348. Min Act: -0.0000

Data Index: 1576265 (The Pile)

Max Activating Token Index: 315

Text #19

Max Range: 1.1854. Min Range: -1.1854

Max Act: 0.7194. Min Act: -0.0000

Data Index: 179972 (The Pile)

Max Activating Token Index: 842