Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1937 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.4373. Min Act: -0.0000

Data Index: 1142971 (The Pile)

Max Activating Token Index: 26

Text #1

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.2209. Min Act: -0.0000

Data Index: 991414 (The Pile)

Max Activating Token Index: 830

Text #2

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.1684. Min Act: -0.0000

Data Index: 878314 (The Pile)

Max Activating Token Index: 161

Text #3

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.2105. Min Act: -0.0000

Data Index: 603974 (The Pile)

Max Activating Token Index: 786

Text #4

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.1222. Min Act: -0.0000

Data Index: 157521 (The Pile)

Max Activating Token Index: 732

Text #5

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0694. Min Act: -0.0000

Data Index: 606558 (The Pile)

Max Activating Token Index: 537

Text #6

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0956. Min Act: -0.0000

Data Index: 942284 (The Pile)

Max Activating Token Index: 492

Text #7

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0508. Min Act: -0.0000

Data Index: 1021535 (The Pile)

Max Activating Token Index: 474

Text #8

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0136. Min Act: -0.0001

Data Index: 56199 (The Pile)

Max Activating Token Index: 495

Text #9

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.1373. Min Act: -0.0000

Data Index: 741070 (The Pile)

Max Activating Token Index: 614

Text #10

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0929. Min Act: -0.0000

Data Index: 1983786 (The Pile)

Max Activating Token Index: 925

Text #11

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0713. Min Act: -0.0000

Data Index: 1346354 (The Pile)

Max Activating Token Index: 394

Text #12

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9990. Min Act: -0.0000

Data Index: 1879534 (The Pile)

Max Activating Token Index: 588

Text #13

Max Range: 1.4373. Min Range: -1.4373

Max Act: 1.0150. Min Act: -0.0000

Data Index: 1256559 (The Pile)

Max Activating Token Index: 164

Text #14

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9570. Min Act: -0.0000

Data Index: 58206 (The Pile)

Max Activating Token Index: 537

Text #15

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9975. Min Act: -0.0000

Data Index: 1994661 (The Pile)

Max Activating Token Index: 594

Text #16

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9311. Min Act: -0.0000

Data Index: 115562 (The Pile)

Max Activating Token Index: 740

Text #17

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.8943. Min Act: -0.0001

Data Index: 646915 (The Pile)

Max Activating Token Index: 986

Text #18

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9396. Min Act: -0.0000

Data Index: 192933 (The Pile)

Max Activating Token Index: 672

Text #19

Max Range: 1.4373. Min Range: -1.4373

Max Act: 0.9282. Min Act: -0.0000

Data Index: 179893 (The Pile)

Max Activating Token Index: 532