Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2112 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1773. Min Range: -1.1773

Max Act: 1.1379. Min Act: -0.0000

Data Index: 1188565 (The Pile)

Max Activating Token Index: 291

Text #1

Max Range: 1.1773. Min Range: -1.1773

Max Act: 1.1773. Min Act: -0.0000

Data Index: 294710 (The Pile)

Max Activating Token Index: 162

Text #2

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8836. Min Act: -0.0000

Data Index: 625511 (The Pile)

Max Activating Token Index: 107

Text #3

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8678. Min Act: -0.0000

Data Index: 585807 (The Pile)

Max Activating Token Index: 360

Text #4

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8349. Min Act: -0.0000

Data Index: 1376447 (The Pile)

Max Activating Token Index: 734

Text #5

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8106. Min Act: -0.0000

Data Index: 1396955 (The Pile)

Max Activating Token Index: 734

Text #6

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8337. Min Act: -0.0000

Data Index: 876750 (The Pile)

Max Activating Token Index: 52

Text #7

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.8076. Min Act: -0.0000

Data Index: 897131 (The Pile)

Max Activating Token Index: 163

Text #8

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7179. Min Act: -0.0000

Data Index: 773436 (The Pile)

Max Activating Token Index: 422

Text #9

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7580. Min Act: -0.0000

Data Index: 1920608 (The Pile)

Max Activating Token Index: 237

Text #10

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7835. Min Act: -0.0000

Data Index: 370875 (The Pile)

Max Activating Token Index: 914

Text #11

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.6798. Min Act: -0.0000

Data Index: 189400 (The Pile)

Max Activating Token Index: 867

Text #12

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.6879. Min Act: -0.0000

Data Index: 1040428 (The Pile)

Max Activating Token Index: 979

Text #13

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7141. Min Act: -0.0000

Data Index: 729368 (The Pile)

Max Activating Token Index: 558

Text #14

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7262. Min Act: -0.0000

Data Index: 1666557 (The Pile)

Max Activating Token Index: 339

Text #15

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.6208. Min Act: -0.0000

Data Index: 1425752 (The Pile)

Max Activating Token Index: 1007

Text #16

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7230. Min Act: -0.0000

Data Index: 1638431 (The Pile)

Max Activating Token Index: 302

Text #17

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7640. Min Act: -0.0000

Data Index: 247152 (The Pile)

Max Activating Token Index: 670

Text #18

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7138. Min Act: -0.0000

Data Index: 110045 (The Pile)

Max Activating Token Index: 366

Text #19

Max Range: 1.1773. Min Range: -1.1773

Max Act: 0.7005. Min Act: -0.0000

Data Index: 318015 (The Pile)

Max Activating Token Index: 46