Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1226 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1244. Min Range: -3.1244

Max Act: 3.1244. Min Act: -0.0000

Data Index: 315228 (The Pile)

Max Activating Token Index: 643

Text #1

Max Range: 3.1244. Min Range: -3.1244

Max Act: 3.0742. Min Act: -0.0000

Data Index: 468357 (The Pile)

Max Activating Token Index: 591

Text #2

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.9682. Min Act: -0.0000

Data Index: 472157 (The Pile)

Max Activating Token Index: 27

Text #3

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.6668. Min Act: -0.0000

Data Index: 348581 (The Pile)

Max Activating Token Index: 784

Text #4

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.9067. Min Act: -0.0000

Data Index: 509323 (The Pile)

Max Activating Token Index: 574

Text #5

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.8984. Min Act: -0.0000

Data Index: 1600127 (The Pile)

Max Activating Token Index: 156

Text #6

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.9951. Min Act: -0.0000

Data Index: 1028708 (The Pile)

Max Activating Token Index: 697

Text #7

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.8033. Min Act: -0.0000

Data Index: 1601476 (The Pile)

Max Activating Token Index: 36

Text #8

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.6392. Min Act: -0.0000

Data Index: 1853761 (The Pile)

Max Activating Token Index: 533

Text #9

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.3965. Min Act: -0.0000

Data Index: 1782385 (The Pile)

Max Activating Token Index: 342

Text #10

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.6600. Min Act: -0.0000

Data Index: 1028111 (The Pile)

Max Activating Token Index: 39

Text #11

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.2891. Min Act: -0.0000

Data Index: 536025 (The Pile)

Max Activating Token Index: 1004

Text #12

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.4694. Min Act: -0.0000

Data Index: 706624 (The Pile)

Max Activating Token Index: 35

Text #13

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.7085. Min Act: -0.0000

Data Index: 1173859 (The Pile)

Max Activating Token Index: 990

Text #14

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.2663. Min Act: -0.0000

Data Index: 1438277 (The Pile)

Max Activating Token Index: 771

Text #15

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.6332. Min Act: -0.0000

Data Index: 179018 (The Pile)

Max Activating Token Index: 940

Text #16

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.3971. Min Act: -0.0000

Data Index: 1609851 (The Pile)

Max Activating Token Index: 910

Text #17

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.4613. Min Act: -0.0000

Data Index: 1606970 (The Pile)

Max Activating Token Index: 585

Text #18

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.2940. Min Act: -0.0000

Data Index: 1006364 (The Pile)

Max Activating Token Index: 388

Text #19

Max Range: 3.1244. Min Range: -3.1244

Max Act: 2.4659. Min Act: -0.0000

Data Index: 1974455 (The Pile)

Max Activating Token Index: 308