Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3591 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.3707. Min Act: -0.0000

Data Index: 976200 (The Pile)

Max Activating Token Index: 147

Text #1

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.2826. Min Act: -0.0000

Data Index: 1457481 (The Pile)

Max Activating Token Index: 105

Text #2

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.3066. Min Act: -0.0000

Data Index: 861875 (The Pile)

Max Activating Token Index: 644

Text #3

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.5364. Min Act: -0.0000

Data Index: 107963 (The Pile)

Max Activating Token Index: 1012

Text #4

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.0191. Min Act: -0.0000

Data Index: 1518345 (The Pile)

Max Activating Token Index: 645

Text #5

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.7805. Min Act: -0.0000

Data Index: 588342 (The Pile)

Max Activating Token Index: 732

Text #6

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.1531. Min Act: -0.0000

Data Index: 182034 (The Pile)

Max Activating Token Index: 707

Text #7

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.1187. Min Act: -0.0000

Data Index: 309426 (The Pile)

Max Activating Token Index: 911

Text #8

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.9106. Min Act: -0.0000

Data Index: 695212 (The Pile)

Max Activating Token Index: 555

Text #9

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.2289. Min Act: -0.0000

Data Index: 99449 (The Pile)

Max Activating Token Index: 408

Text #10

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.8679. Min Act: -0.0000

Data Index: 669459 (The Pile)

Max Activating Token Index: 156

Text #11

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.1364. Min Act: -0.0000

Data Index: 101498 (The Pile)

Max Activating Token Index: 549

Text #12

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.7578. Min Act: -0.0000

Data Index: 1868602 (The Pile)

Max Activating Token Index: 85

Text #13

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.5310. Min Act: -0.0000

Data Index: 1243065 (The Pile)

Max Activating Token Index: 681

Text #14

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.8428. Min Act: -0.0000

Data Index: 1890902 (The Pile)

Max Activating Token Index: 341

Text #15

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.5169. Min Act: -0.0000

Data Index: 1117301 (The Pile)

Max Activating Token Index: 94

Text #16

Max Range: 4.5364. Min Range: -4.5364

Max Act: 4.0243. Min Act: -0.0000

Data Index: 1205498 (The Pile)

Max Activating Token Index: 895

Text #17

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.8492. Min Act: -0.0000

Data Index: 608361 (The Pile)

Max Activating Token Index: 680

Text #18

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.6680. Min Act: -0.0000

Data Index: 716851 (The Pile)

Max Activating Token Index: 244

Text #19

Max Range: 4.5364. Min Range: -4.5364

Max Act: 3.9761. Min Act: -0.0000

Data Index: 184100 (The Pile)

Max Activating Token Index: 782