Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 35 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1607. Min Act: -0.0000

Data Index: 20021 (The Pile)

Max Activating Token Index: 531

Text #1

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1680. Min Act: -0.0000

Data Index: 817763 (The Pile)

Max Activating Token Index: 381

Text #2

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1014. Min Act: -0.0000

Data Index: 1932226 (The Pile)

Max Activating Token Index: 428

Text #3

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1102. Min Act: -0.0000

Data Index: 697317 (The Pile)

Max Activating Token Index: 925

Text #4

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0698. Min Act: -0.0000

Data Index: 466985 (The Pile)

Max Activating Token Index: 809

Text #5

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1214. Min Act: -0.0000

Data Index: 1562730 (The Pile)

Max Activating Token Index: 43

Text #6

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1169. Min Act: -0.0000

Data Index: 1548880 (The Pile)

Max Activating Token Index: 389

Text #7

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0389. Min Act: -0.0000

Data Index: 1284535 (The Pile)

Max Activating Token Index: 646

Text #8

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0689. Min Act: -0.0000

Data Index: 915669 (The Pile)

Max Activating Token Index: 569

Text #9

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0899. Min Act: -0.0000

Data Index: 655554 (The Pile)

Max Activating Token Index: 481

Text #10

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0470. Min Act: -0.0000

Data Index: 1876847 (The Pile)

Max Activating Token Index: 375

Text #11

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1361. Min Act: -0.0000

Data Index: 873880 (The Pile)

Max Activating Token Index: 163

Text #12

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0348. Min Act: -0.0000

Data Index: 1315665 (The Pile)

Max Activating Token Index: 991

Text #13

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0629. Min Act: -0.0000

Data Index: 796012 (The Pile)

Max Activating Token Index: 112

Text #14

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0320. Min Act: -0.0000

Data Index: 149481 (The Pile)

Max Activating Token Index: 199

Text #15

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0236. Min Act: -0.0000

Data Index: 1122138 (The Pile)

Max Activating Token Index: 529

Text #16

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0754. Min Act: -0.0000

Data Index: 717135 (The Pile)

Max Activating Token Index: 968

Text #17

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0670. Min Act: -0.0000

Data Index: 407082 (The Pile)

Max Activating Token Index: 901

Text #18

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.0111. Min Act: -0.0000

Data Index: 1726570 (The Pile)

Max Activating Token Index: 135

Text #19

Max Range: 1.1680. Min Range: -1.1680

Max Act: 1.1116. Min Act: -0.0000

Data Index: 61719 (The Pile)

Max Activating Token Index: 602