Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4282 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.1450. Min Act: -0.0000

Data Index: 1120465 (The Pile)

Max Activating Token Index: 8

Text #1

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.1819. Min Act: -0.0000

Data Index: 1775813 (The Pile)

Max Activating Token Index: 438

Text #2

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.1029. Min Act: -0.0000

Data Index: 1750370 (The Pile)

Max Activating Token Index: 349

Text #3

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.2169. Min Act: -0.0000

Data Index: 110214 (The Pile)

Max Activating Token Index: 949

Text #4

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.2284. Min Act: -0.0000

Data Index: 1280242 (The Pile)

Max Activating Token Index: 89

Text #5

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0720. Min Act: -0.0000

Data Index: 1959401 (The Pile)

Max Activating Token Index: 177

Text #6

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0357. Min Act: -0.0000

Data Index: 412621 (The Pile)

Max Activating Token Index: 916

Text #7

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9554. Min Act: -0.0000

Data Index: 711079 (The Pile)

Max Activating Token Index: 597

Text #8

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9481. Min Act: -0.0000

Data Index: 1037399 (The Pile)

Max Activating Token Index: 539

Text #9

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9908. Min Act: -0.0000

Data Index: 1902581 (The Pile)

Max Activating Token Index: 844

Text #10

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9418. Min Act: -0.0000

Data Index: 1796388 (The Pile)

Max Activating Token Index: 125

Text #11

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0458. Min Act: -0.0000

Data Index: 1876468 (The Pile)

Max Activating Token Index: 250

Text #12

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0484. Min Act: -0.0000

Data Index: 1946283 (The Pile)

Max Activating Token Index: 42

Text #13

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0399. Min Act: -0.0000

Data Index: 15512 (The Pile)

Max Activating Token Index: 511

Text #14

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0427. Min Act: -0.0000

Data Index: 1564752 (The Pile)

Max Activating Token Index: 280

Text #15

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.8755. Min Act: -0.0000

Data Index: 338387 (The Pile)

Max Activating Token Index: 154

Text #16

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9427. Min Act: -0.0000

Data Index: 1682608 (The Pile)

Max Activating Token Index: 434

Text #17

Max Range: 1.2284. Min Range: -1.2284

Max Act: 1.0524. Min Act: -0.0000

Data Index: 1247222 (The Pile)

Max Activating Token Index: 137

Text #18

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9039. Min Act: -0.0000

Data Index: 1386776 (The Pile)

Max Activating Token Index: 357

Text #19

Max Range: 1.2284. Min Range: -1.2284

Max Act: 0.9496. Min Act: -0.0000

Data Index: 634119 (The Pile)

Max Activating Token Index: 272