Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2641 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.7311. Min Act: -0.0000

Data Index: 1320006 (The Pile)

Max Activating Token Index: 588

Text #1

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.7977. Min Act: -0.0000

Data Index: 691783 (The Pile)

Max Activating Token Index: 782

Text #2

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.8293. Min Act: -0.0000

Data Index: 398479 (The Pile)

Max Activating Token Index: 600

Text #3

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6003. Min Act: -0.0000

Data Index: 1207776 (The Pile)

Max Activating Token Index: 875

Text #4

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6003. Min Act: -0.0000

Data Index: 1880005 (The Pile)

Max Activating Token Index: 494

Text #5

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6096. Min Act: -0.0000

Data Index: 1267164 (The Pile)

Max Activating Token Index: 187

Text #6

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.8235. Min Act: -0.0000

Data Index: 540009 (The Pile)

Max Activating Token Index: 524

Text #7

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.5935. Min Act: -0.0000

Data Index: 831129 (The Pile)

Max Activating Token Index: 187

Text #8

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6352. Min Act: -0.0000

Data Index: 599316 (The Pile)

Max Activating Token Index: 341

Text #9

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6021. Min Act: -0.0000

Data Index: 1879297 (The Pile)

Max Activating Token Index: 737

Text #10

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6489. Min Act: -0.0000

Data Index: 972618 (The Pile)

Max Activating Token Index: 283

Text #11

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.7475. Min Act: -0.0000

Data Index: 923072 (The Pile)

Max Activating Token Index: 448

Text #12

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.5760. Min Act: -0.0000

Data Index: 1572684 (The Pile)

Max Activating Token Index: 721

Text #13

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6398. Min Act: -0.0000

Data Index: 1924471 (The Pile)

Max Activating Token Index: 244

Text #14

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6058. Min Act: -0.0000

Data Index: 1187105 (The Pile)

Max Activating Token Index: 347

Text #15

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.5285. Min Act: -0.0000

Data Index: 656896 (The Pile)

Max Activating Token Index: 917

Text #16

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.3917. Min Act: -0.0000

Data Index: 792222 (The Pile)

Max Activating Token Index: 644

Text #17

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.5664. Min Act: -0.0000

Data Index: 1687891 (The Pile)

Max Activating Token Index: 793

Text #18

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6653. Min Act: -0.0000

Data Index: 1378312 (The Pile)

Max Activating Token Index: 286

Text #19

Max Range: 1.8293. Min Range: -1.8293

Max Act: 1.6580. Min Act: -0.0000

Data Index: 359328 (The Pile)

Max Activating Token Index: 660