Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3271 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.8172. Min Act: -0.0000

Data Index: 1050929 (The Pile)

Max Activating Token Index: 697

Text #1

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.6789. Min Act: -0.0000

Data Index: 1723317 (The Pile)

Max Activating Token Index: 27

Text #2

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.6283. Min Act: -0.0000

Data Index: 1467095 (The Pile)

Max Activating Token Index: 818

Text #3

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.6645. Min Act: -0.0000

Data Index: 1909953 (The Pile)

Max Activating Token Index: 224

Text #4

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.6679. Min Act: -0.0000

Data Index: 742742 (The Pile)

Max Activating Token Index: 163

Text #5

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5894. Min Act: -0.0000

Data Index: 1888634 (The Pile)

Max Activating Token Index: 251

Text #6

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5868. Min Act: -0.0000

Data Index: 1949521 (The Pile)

Max Activating Token Index: 263

Text #7

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5361. Min Act: -0.0000

Data Index: 1325919 (The Pile)

Max Activating Token Index: 425

Text #8

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5479. Min Act: -0.0000

Data Index: 1770076 (The Pile)

Max Activating Token Index: 86

Text #9

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5387. Min Act: -0.0000

Data Index: 1668857 (The Pile)

Max Activating Token Index: 984

Text #10

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5390. Min Act: -0.0000

Data Index: 1773324 (The Pile)

Max Activating Token Index: 255

Text #11

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5451. Min Act: -0.0000

Data Index: 1982693 (The Pile)

Max Activating Token Index: 928

Text #12

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.4995. Min Act: -0.0000

Data Index: 1329231 (The Pile)

Max Activating Token Index: 720

Text #13

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.4791. Min Act: -0.0000

Data Index: 1805089 (The Pile)

Max Activating Token Index: 801

Text #14

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5226. Min Act: -0.0000

Data Index: 819796 (The Pile)

Max Activating Token Index: 556

Text #15

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.4830. Min Act: -0.0000

Data Index: 792724 (The Pile)

Max Activating Token Index: 546

Text #16

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.4778. Min Act: -0.0000

Data Index: 1110816 (The Pile)

Max Activating Token Index: 765

Text #17

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5152. Min Act: -0.0000

Data Index: 1267397 (The Pile)

Max Activating Token Index: 407

Text #18

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.5080. Min Act: -0.0000

Data Index: 1599347 (The Pile)

Max Activating Token Index: 274

Text #19

Max Range: 0.8172. Min Range: -0.8172

Max Act: 0.4742. Min Act: -0.0000

Data Index: 133588 (The Pile)

Max Activating Token Index: 284