Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 439 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.6997. Min Act: -0.0000

Data Index: 1469314 (The Pile)

Max Activating Token Index: 570

Text #1

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.5282. Min Act: -0.0000

Data Index: 665408 (The Pile)

Max Activating Token Index: 960

Text #2

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.5229. Min Act: -0.0000

Data Index: 1338668 (The Pile)

Max Activating Token Index: 654

Text #3

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.4108. Min Act: -0.0000

Data Index: 1380357 (The Pile)

Max Activating Token Index: 553

Text #4

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.3883. Min Act: -0.0000

Data Index: 1393775 (The Pile)

Max Activating Token Index: 128

Text #5

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.3155. Min Act: -0.0000

Data Index: 1757231 (The Pile)

Max Activating Token Index: 112

Text #6

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.3223. Min Act: -0.0000

Data Index: 55531 (The Pile)

Max Activating Token Index: 945

Text #7

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.3205. Min Act: -0.0000

Data Index: 763856 (The Pile)

Max Activating Token Index: 213

Text #8

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.1479. Min Act: -0.0000

Data Index: 1211378 (The Pile)

Max Activating Token Index: 553

Text #9

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2818. Min Act: -0.0000

Data Index: 642541 (The Pile)

Max Activating Token Index: 148

Text #10

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2612. Min Act: -0.0000

Data Index: 478016 (The Pile)

Max Activating Token Index: 488

Text #11

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.1836. Min Act: -0.0000

Data Index: 401394 (The Pile)

Max Activating Token Index: 216

Text #12

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.1697. Min Act: -0.0000

Data Index: 1481021 (The Pile)

Max Activating Token Index: 345

Text #13

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2295. Min Act: -0.0000

Data Index: 877264 (The Pile)

Max Activating Token Index: 33

Text #14

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.1474. Min Act: -0.0000

Data Index: 1181898 (The Pile)

Max Activating Token Index: 770

Text #15

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2152. Min Act: -0.0000

Data Index: 1628922 (The Pile)

Max Activating Token Index: 535

Text #16

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2057. Min Act: -0.0000

Data Index: 1021490 (The Pile)

Max Activating Token Index: 483

Text #17

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2188. Min Act: -0.0000

Data Index: 651098 (The Pile)

Max Activating Token Index: 561

Text #18

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2140. Min Act: -0.0000

Data Index: 1867487 (The Pile)

Max Activating Token Index: 42

Text #19

Max Range: 1.6997. Min Range: -1.6997

Max Act: 1.2053. Min Act: -0.0000

Data Index: 1239992 (The Pile)

Max Activating Token Index: 846