Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1691 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.7074. Min Act: -0.0000

Data Index: 1569404 (The Pile)

Max Activating Token Index: 418

Text #1

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6571. Min Act: -0.0000

Data Index: 1976100 (The Pile)

Max Activating Token Index: 543

Text #2

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6334. Min Act: -0.0000

Data Index: 265831 (The Pile)

Max Activating Token Index: 916

Text #3

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5931. Min Act: -0.0000

Data Index: 1853470 (The Pile)

Max Activating Token Index: 562

Text #4

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5553. Min Act: -0.0000

Data Index: 1446308 (The Pile)

Max Activating Token Index: 552

Text #5

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6549. Min Act: -0.0000

Data Index: 860068 (The Pile)

Max Activating Token Index: 729

Text #6

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6177. Min Act: -0.0000

Data Index: 513047 (The Pile)

Max Activating Token Index: 119

Text #7

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5521. Min Act: -0.0000

Data Index: 1323586 (The Pile)

Max Activating Token Index: 282

Text #8

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6053. Min Act: -0.0000

Data Index: 568457 (The Pile)

Max Activating Token Index: 215

Text #9

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.6015. Min Act: -0.0000

Data Index: 118815 (The Pile)

Max Activating Token Index: 161

Text #10

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5961. Min Act: -0.0000

Data Index: 96157 (The Pile)

Max Activating Token Index: 158

Text #11

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.4669. Min Act: -0.0000

Data Index: 1059075 (The Pile)

Max Activating Token Index: 747

Text #12

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5378. Min Act: -0.0000

Data Index: 1444893 (The Pile)

Max Activating Token Index: 361

Text #13

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5365. Min Act: -0.0000

Data Index: 1305635 (The Pile)

Max Activating Token Index: 382

Text #14

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5160. Min Act: -0.0000

Data Index: 1294640 (The Pile)

Max Activating Token Index: 464

Text #15

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5458. Min Act: -0.0000

Data Index: 1261988 (The Pile)

Max Activating Token Index: 839

Text #16

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5655. Min Act: -0.0000

Data Index: 1295488 (The Pile)

Max Activating Token Index: 84

Text #17

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5410. Min Act: -0.0000

Data Index: 1613022 (The Pile)

Max Activating Token Index: 327

Text #18

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5105. Min Act: -0.0000

Data Index: 758465 (The Pile)

Max Activating Token Index: 703

Text #19

Max Range: 0.7074. Min Range: -0.7074

Max Act: 0.5037. Min Act: -0.0000

Data Index: 424722 (The Pile)

Max Activating Token Index: 234