Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2668 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.4179. Min Act: -0.0000

Data Index: 508029 (The Pile)

Max Activating Token Index: 851

Text #1

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.3407. Min Act: -0.0000

Data Index: 634633 (The Pile)

Max Activating Token Index: 546

Text #2

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.3133. Min Act: -0.0000

Data Index: 1810270 (The Pile)

Max Activating Token Index: 588

Text #3

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.3626. Min Act: -0.0000

Data Index: 845322 (The Pile)

Max Activating Token Index: 928

Text #4

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2924. Min Act: -0.0000

Data Index: 81155 (The Pile)

Max Activating Token Index: 1003

Text #5

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.3069. Min Act: -0.0000

Data Index: 1103031 (The Pile)

Max Activating Token Index: 882

Text #6

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2873. Min Act: -0.0000

Data Index: 8621 (The Pile)

Max Activating Token Index: 584

Text #7

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2815. Min Act: -0.0000

Data Index: 1859183 (The Pile)

Max Activating Token Index: 637

Text #8

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2838. Min Act: -0.0000

Data Index: 728479 (The Pile)

Max Activating Token Index: 210

Text #9

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.3028. Min Act: -0.0000

Data Index: 807698 (The Pile)

Max Activating Token Index: 626

Text #10

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2024. Min Act: -0.0000

Data Index: 553132 (The Pile)

Max Activating Token Index: 1011

Text #11

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2544. Min Act: -0.0000

Data Index: 391905 (The Pile)

Max Activating Token Index: 592

Text #12

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2583. Min Act: -0.0000

Data Index: 1338175 (The Pile)

Max Activating Token Index: 928

Text #13

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2538. Min Act: -0.0000

Data Index: 1495313 (The Pile)

Max Activating Token Index: 679

Text #14

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2449. Min Act: -0.0000

Data Index: 927719 (The Pile)

Max Activating Token Index: 801

Text #15

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2182. Min Act: -0.0000

Data Index: 1577584 (The Pile)

Max Activating Token Index: 1010

Text #16

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2187. Min Act: -0.0000

Data Index: 1411196 (The Pile)

Max Activating Token Index: 472

Text #17

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2341. Min Act: -0.0000

Data Index: 1924988 (The Pile)

Max Activating Token Index: 322

Text #18

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.1915. Min Act: -0.0000

Data Index: 1018397 (The Pile)

Max Activating Token Index: 680

Text #19

Max Range: 0.4179. Min Range: -0.4179

Max Act: 0.2215. Min Act: -0.0000

Data Index: 1857560 (The Pile)

Max Activating Token Index: 152