Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1370 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.4013. Min Act: -0.0000

Data Index: 1616835 (The Pile)

Max Activating Token Index: 789

Text #1

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.4378. Min Act: -0.0000

Data Index: 1510369 (The Pile)

Max Activating Token Index: 490

Text #2

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.4605. Min Act: -0.0000

Data Index: 1232252 (The Pile)

Max Activating Token Index: 161

Text #3

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.3415. Min Act: -0.0000

Data Index: 1243055 (The Pile)

Max Activating Token Index: 58

Text #4

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1234. Min Act: -0.0000

Data Index: 1050974 (The Pile)

Max Activating Token Index: 356

Text #5

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2519. Min Act: -0.0000

Data Index: 973058 (The Pile)

Max Activating Token Index: 201

Text #6

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2311. Min Act: -0.0000

Data Index: 573557 (The Pile)

Max Activating Token Index: 552

Text #7

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2804. Min Act: -0.0000

Data Index: 1263506 (The Pile)

Max Activating Token Index: 771

Text #8

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.3382. Min Act: -0.0000

Data Index: 858631 (The Pile)

Max Activating Token Index: 219

Text #9

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1477. Min Act: -0.0000

Data Index: 783362 (The Pile)

Max Activating Token Index: 899

Text #10

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2372. Min Act: -0.0000

Data Index: 837373 (The Pile)

Max Activating Token Index: 39

Text #11

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2135. Min Act: -0.0000

Data Index: 612841 (The Pile)

Max Activating Token Index: 693

Text #12

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2441. Min Act: -0.0000

Data Index: 1422973 (The Pile)

Max Activating Token Index: 244

Text #13

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.0577. Min Act: -0.0000

Data Index: 1291946 (The Pile)

Max Activating Token Index: 80

Text #14

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.2280. Min Act: -0.0000

Data Index: 956348 (The Pile)

Max Activating Token Index: 120

Text #15

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1027. Min Act: -0.0000

Data Index: 1739794 (The Pile)

Max Activating Token Index: 80

Text #16

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1068. Min Act: -0.0000

Data Index: 1618489 (The Pile)

Max Activating Token Index: 644

Text #17

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1354. Min Act: -0.0000

Data Index: 1866677 (The Pile)

Max Activating Token Index: 290

Text #18

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1956. Min Act: -0.0000

Data Index: 90970 (The Pile)

Max Activating Token Index: 22

Text #19

Max Range: 1.4605. Min Range: -1.4605

Max Act: 1.1976. Min Act: -0.0000

Data Index: 1868709 (The Pile)

Max Activating Token Index: 440