Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4432 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.4575. Min Act: -0.0000

Data Index: 179734 (The Pile)

Max Activating Token Index: 457

Text #1

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.3197. Min Act: -0.0000

Data Index: 730054 (The Pile)

Max Activating Token Index: 403

Text #2

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.3620. Min Act: -0.0000

Data Index: 741308 (The Pile)

Max Activating Token Index: 162

Text #3

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.3190. Min Act: -0.0000

Data Index: 1991295 (The Pile)

Max Activating Token Index: 226

Text #4

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2655. Min Act: -0.0000

Data Index: 1220364 (The Pile)

Max Activating Token Index: 47

Text #5

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2626. Min Act: -0.0000

Data Index: 290465 (The Pile)

Max Activating Token Index: 632

Text #6

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2390. Min Act: -0.0000

Data Index: 1272593 (The Pile)

Max Activating Token Index: 308

Text #7

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2840. Min Act: -0.0000

Data Index: 1272344 (The Pile)

Max Activating Token Index: 312

Text #8

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2071. Min Act: -0.0000

Data Index: 1222200 (The Pile)

Max Activating Token Index: 785

Text #9

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1228. Min Act: -0.0000

Data Index: 590609 (The Pile)

Max Activating Token Index: 951

Text #10

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1981. Min Act: -0.0000

Data Index: 587777 (The Pile)

Max Activating Token Index: 458

Text #11

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2271. Min Act: -0.0000

Data Index: 507163 (The Pile)

Max Activating Token Index: 105

Text #12

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1365. Min Act: -0.0000

Data Index: 498280 (The Pile)

Max Activating Token Index: 377

Text #13

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.0731. Min Act: -0.0000

Data Index: 1944419 (The Pile)

Max Activating Token Index: 813

Text #14

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1266. Min Act: -0.0000

Data Index: 1450148 (The Pile)

Max Activating Token Index: 828

Text #15

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.2014. Min Act: -0.0000

Data Index: 1376459 (The Pile)

Max Activating Token Index: 585

Text #16

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1051. Min Act: -0.0000

Data Index: 491968 (The Pile)

Max Activating Token Index: 169

Text #17

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.0469. Min Act: -0.0000

Data Index: 819251 (The Pile)

Max Activating Token Index: 222

Text #18

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.1058. Min Act: -0.0000

Data Index: 1763460 (The Pile)

Max Activating Token Index: 997

Text #19

Max Range: 1.4575. Min Range: -1.4575

Max Act: 1.0339. Min Act: -0.0000

Data Index: 1268993 (The Pile)

Max Activating Token Index: 502