Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1582 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.6224. Min Act: -0.0000

Data Index: 1588841 (The Pile)

Max Activating Token Index: 97

Text #1

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.6582. Min Act: 0.0000

Data Index: 1307386 (The Pile)

Max Activating Token Index: 323

Text #2

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.4449. Min Act: 0.0000

Data Index: 1512743 (The Pile)

Max Activating Token Index: 105

Text #3

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.4707. Min Act: -0.0000

Data Index: 299621 (The Pile)

Max Activating Token Index: 42

Text #4

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.4099. Min Act: -0.0000

Data Index: 1515801 (The Pile)

Max Activating Token Index: 370

Text #5

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2413. Min Act: -0.0000

Data Index: 1604642 (The Pile)

Max Activating Token Index: 257

Text #6

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.4022. Min Act: 0.0000

Data Index: 534619 (The Pile)

Max Activating Token Index: 553

Text #7

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2264. Min Act: -0.0000

Data Index: 1932831 (The Pile)

Max Activating Token Index: 714

Text #8

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2302. Min Act: 0.0000

Data Index: 1530258 (The Pile)

Max Activating Token Index: 20

Text #9

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2494. Min Act: -0.0000

Data Index: 100710 (The Pile)

Max Activating Token Index: 207

Text #10

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2443. Min Act: -0.0000

Data Index: 192037 (The Pile)

Max Activating Token Index: 726

Text #11

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.2475. Min Act: -0.0000

Data Index: 1652494 (The Pile)

Max Activating Token Index: 128

Text #12

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1979. Min Act: -0.0000

Data Index: 649458 (The Pile)

Max Activating Token Index: 869

Text #13

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1145. Min Act: -0.0000

Data Index: 1386318 (The Pile)

Max Activating Token Index: 452

Text #14

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1287. Min Act: -0.0000

Data Index: 919098 (The Pile)

Max Activating Token Index: 693

Text #15

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1741. Min Act: -0.0000

Data Index: 1101015 (The Pile)

Max Activating Token Index: 1005

Text #16

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.1109. Min Act: -0.0000

Data Index: 154902 (The Pile)

Max Activating Token Index: 137

Text #17

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.0659. Min Act: -0.0000

Data Index: 1152433 (The Pile)

Max Activating Token Index: 410

Text #18

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.0785. Min Act: -0.0000

Data Index: 1944128 (The Pile)

Max Activating Token Index: 805

Text #19

Max Range: 1.6582. Min Range: -1.6582

Max Act: 1.0692. Min Act: -0.0000

Data Index: 92092 (The Pile)

Max Activating Token Index: 417