Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4148 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.9326. Min Act: -0.0000

Data Index: 1309791 (The Pile)

Max Activating Token Index: 660

Text #1

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.8269. Min Act: -0.0000

Data Index: 1544760 (The Pile)

Max Activating Token Index: 364

Text #2

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.3021. Min Act: -0.0000

Data Index: 993193 (The Pile)

Max Activating Token Index: 266

Text #3

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2861. Min Act: -0.0000

Data Index: 461713 (The Pile)

Max Activating Token Index: 195

Text #4

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2598. Min Act: -0.0000

Data Index: 1908336 (The Pile)

Max Activating Token Index: 577

Text #5

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2585. Min Act: -0.0000

Data Index: 1269668 (The Pile)

Max Activating Token Index: 352

Text #6

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1782. Min Act: -0.0000

Data Index: 394308 (The Pile)

Max Activating Token Index: 76

Text #7

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2459. Min Act: -0.0000

Data Index: 1529659 (The Pile)

Max Activating Token Index: 579

Text #8

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2192. Min Act: -0.0000

Data Index: 1283431 (The Pile)

Max Activating Token Index: 264

Text #9

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1425. Min Act: -0.0000

Data Index: 1889081 (The Pile)

Max Activating Token Index: 904

Text #10

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2096. Min Act: -0.0000

Data Index: 1833544 (The Pile)

Max Activating Token Index: 575

Text #11

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2460. Min Act: -0.0000

Data Index: 542959 (The Pile)

Max Activating Token Index: 119

Text #12

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1407. Min Act: -0.0000

Data Index: 1858886 (The Pile)

Max Activating Token Index: 505

Text #13

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2638. Min Act: -0.0000

Data Index: 1389883 (The Pile)

Max Activating Token Index: 186

Text #14

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2702. Min Act: -0.0000

Data Index: 318896 (The Pile)

Max Activating Token Index: 663

Text #15

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1904. Min Act: -0.0000

Data Index: 574435 (The Pile)

Max Activating Token Index: 182

Text #16

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1692. Min Act: -0.0000

Data Index: 1744522 (The Pile)

Max Activating Token Index: 250

Text #17

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.1457. Min Act: -0.0000

Data Index: 1274070 (The Pile)

Max Activating Token Index: 127

Text #18

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2140. Min Act: -0.0000

Data Index: 1385268 (The Pile)

Max Activating Token Index: 1012

Text #19

Max Range: 1.9326. Min Range: -1.9326

Max Act: 1.2061. Min Act: -0.0000

Data Index: 442495 (The Pile)

Max Activating Token Index: 507