Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1677 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2580. Min Range: -1.2580

Max Act: 1.2580. Min Act: -0.0000

Data Index: 73297 (The Pile)

Max Activating Token Index: 452

Text #1

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.7892. Min Act: -0.0000

Data Index: 102349 (The Pile)

Max Activating Token Index: 824

Text #2

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.7987. Min Act: -0.0000

Data Index: 1782918 (The Pile)

Max Activating Token Index: 551

Text #3

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.7285. Min Act: -0.0000

Data Index: 847961 (The Pile)

Max Activating Token Index: 629

Text #4

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.6982. Min Act: -0.0000

Data Index: 763334 (The Pile)

Max Activating Token Index: 569

Text #5

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.7123. Min Act: -0.0000

Data Index: 1288354 (The Pile)

Max Activating Token Index: 936

Text #6

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5906. Min Act: -0.0000

Data Index: 104370 (The Pile)

Max Activating Token Index: 185

Text #7

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.6081. Min Act: -0.0000

Data Index: 1458691 (The Pile)

Max Activating Token Index: 897

Text #8

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5967. Min Act: 0.0000

Data Index: 1146205 (The Pile)

Max Activating Token Index: 49

Text #9

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5437. Min Act: -0.0000

Data Index: 1728381 (The Pile)

Max Activating Token Index: 432

Text #10

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5656. Min Act: -0.0000

Data Index: 1615882 (The Pile)

Max Activating Token Index: 784

Text #11

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4830. Min Act: -0.0000

Data Index: 1344952 (The Pile)

Max Activating Token Index: 539

Text #12

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4253. Min Act: -0.0000

Data Index: 1608702 (The Pile)

Max Activating Token Index: 947

Text #13

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4433. Min Act: -0.0000

Data Index: 537066 (The Pile)

Max Activating Token Index: 1015

Text #14

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5272. Min Act: -0.0000

Data Index: 1683008 (The Pile)

Max Activating Token Index: 46

Text #15

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4778. Min Act: -0.0000

Data Index: 1588180 (The Pile)

Max Activating Token Index: 686

Text #16

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.5024. Min Act: -0.0000

Data Index: 1486750 (The Pile)

Max Activating Token Index: 131

Text #17

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4737. Min Act: -0.0000

Data Index: 1495968 (The Pile)

Max Activating Token Index: 356

Text #18

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4186. Min Act: -0.0000

Data Index: 1419949 (The Pile)

Max Activating Token Index: 810

Text #19

Max Range: 1.2580. Min Range: -1.2580

Max Act: 0.4636. Min Act: -0.0000

Data Index: 1577666 (The Pile)

Max Activating Token Index: 674