Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1779 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.2228. Min Range: -3.2228

Max Act: 3.2228. Min Act: -0.0000

Data Index: 1780577 (The Pile)

Max Activating Token Index: 716

Text #1

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.9290. Min Act: -0.0000

Data Index: 175825 (The Pile)

Max Activating Token Index: 296

Text #2

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.5228. Min Act: -0.0000

Data Index: 1857008 (The Pile)

Max Activating Token Index: 488

Text #3

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.3762. Min Act: -0.0000

Data Index: 1286733 (The Pile)

Max Activating Token Index: 884

Text #4

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.1398. Min Act: -0.0000

Data Index: 92930 (The Pile)

Max Activating Token Index: 1018

Text #5

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.2007. Min Act: -0.0000

Data Index: 560274 (The Pile)

Max Activating Token Index: 203

Text #6

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.0533. Min Act: -0.0000

Data Index: 1042069 (The Pile)

Max Activating Token Index: 816

Text #7

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.0346. Min Act: -0.0000

Data Index: 1584960 (The Pile)

Max Activating Token Index: 813

Text #8

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.0052. Min Act: 0.0001

Data Index: 1235052 (The Pile)

Max Activating Token Index: 484

Text #9

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.0075. Min Act: -0.0000

Data Index: 553310 (The Pile)

Max Activating Token Index: 419

Text #10

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9752. Min Act: -0.0000

Data Index: 1745331 (The Pile)

Max Activating Token Index: 814

Text #11

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.8695. Min Act: -0.0000

Data Index: 1530393 (The Pile)

Max Activating Token Index: 751

Text #12

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9516. Min Act: -0.0000

Data Index: 794157 (The Pile)

Max Activating Token Index: 198

Text #13

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9609. Min Act: 0.0001

Data Index: 506996 (The Pile)

Max Activating Token Index: 388

Text #14

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.8406. Min Act: -0.0000

Data Index: 1189260 (The Pile)

Max Activating Token Index: 914

Text #15

Max Range: 3.2228. Min Range: -3.2228

Max Act: 2.0448. Min Act: -0.0000

Data Index: 1514677 (The Pile)

Max Activating Token Index: 484

Text #16

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.8301. Min Act: -0.0000

Data Index: 447759 (The Pile)

Max Activating Token Index: 820

Text #17

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9304. Min Act: 0.0000

Data Index: 1095507 (The Pile)

Max Activating Token Index: 529

Text #18

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9314. Min Act: -0.0000

Data Index: 25958 (The Pile)

Max Activating Token Index: 326

Text #19

Max Range: 3.2228. Min Range: -3.2228

Max Act: 1.9247. Min Act: 0.0003

Data Index: 1355472 (The Pile)

Max Activating Token Index: 932