Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2976 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.9075. Min Act: -0.0000

Data Index: 1504942 (The Pile)

Max Activating Token Index: 255

Text #1

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7838. Min Act: -0.0000

Data Index: 980409 (The Pile)

Max Activating Token Index: 249

Text #2

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7727. Min Act: -0.0000

Data Index: 1700075 (The Pile)

Max Activating Token Index: 224

Text #3

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7541. Min Act: -0.0000

Data Index: 932155 (The Pile)

Max Activating Token Index: 119

Text #4

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7231. Min Act: -0.0000

Data Index: 345581 (The Pile)

Max Activating Token Index: 66

Text #5

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7114. Min Act: -0.0000

Data Index: 355418 (The Pile)

Max Activating Token Index: 329

Text #6

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.7150. Min Act: -0.0000

Data Index: 1735615 (The Pile)

Max Activating Token Index: 346

Text #7

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6739. Min Act: -0.0000

Data Index: 1549216 (The Pile)

Max Activating Token Index: 163

Text #8

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6686. Min Act: -0.0000

Data Index: 1794300 (The Pile)

Max Activating Token Index: 435

Text #9

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6817. Min Act: -0.0000

Data Index: 1713108 (The Pile)

Max Activating Token Index: 530

Text #10

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6524. Min Act: -0.0000

Data Index: 1129219 (The Pile)

Max Activating Token Index: 604

Text #11

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6937. Min Act: -0.0000

Data Index: 682532 (The Pile)

Max Activating Token Index: 179

Text #12

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6635. Min Act: -0.0000

Data Index: 875940 (The Pile)

Max Activating Token Index: 21

Text #13

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6803. Min Act: -0.0000

Data Index: 65510 (The Pile)

Max Activating Token Index: 261

Text #14

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6528. Min Act: -0.0000

Data Index: 1077735 (The Pile)

Max Activating Token Index: 1009

Text #15

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6291. Min Act: -0.0000

Data Index: 1578168 (The Pile)

Max Activating Token Index: 552

Text #16

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6088. Min Act: -0.0000

Data Index: 1827308 (The Pile)

Max Activating Token Index: 274

Text #17

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6287. Min Act: -0.0000

Data Index: 562134 (The Pile)

Max Activating Token Index: 894

Text #18

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6542. Min Act: -0.0000

Data Index: 148619 (The Pile)

Max Activating Token Index: 534

Text #19

Max Range: 0.9075. Min Range: -0.9075

Max Act: 0.6491. Min Act: -0.0000

Data Index: 694646 (The Pile)

Max Activating Token Index: 279