Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2283 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.9033. Min Act: -0.0000

Data Index: 1610486 (Python Code)

Max Activating Token Index: 759

Text #1

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.8266. Min Act: -0.0000

Data Index: 1428484 (Python Code)

Max Activating Token Index: 73

Text #2

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7898. Min Act: -0.0000

Data Index: 1615105 (Python Code)

Max Activating Token Index: 138

Text #3

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7604. Min Act: -0.0000

Data Index: 1544485 (Python Code)

Max Activating Token Index: 232

Text #4

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7556. Min Act: -0.0000

Data Index: 1545485 (Python Code)

Max Activating Token Index: 79

Text #5

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7618. Min Act: -0.0000

Data Index: 1439979 (Python Code)

Max Activating Token Index: 448

Text #6

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7620. Min Act: -0.0000

Data Index: 1480046 (Python Code)

Max Activating Token Index: 102

Text #7

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6901. Min Act: -0.0000

Data Index: 1559377 (Python Code)

Max Activating Token Index: 699

Text #8

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6983. Min Act: -0.0000

Data Index: 1516581 (Python Code)

Max Activating Token Index: 516

Text #9

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.7069. Min Act: -0.0000

Data Index: 1536653 (Python Code)

Max Activating Token Index: 312

Text #10

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6765. Min Act: -0.0000

Data Index: 1395129 (Python Code)

Max Activating Token Index: 459

Text #11

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6801. Min Act: -0.0000

Data Index: 1581509 (Python Code)

Max Activating Token Index: 826

Text #12

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6775. Min Act: -0.0000

Data Index: 1387360 (Python Code)

Max Activating Token Index: 53

Text #13

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6166. Min Act: -0.0000

Data Index: 1646268 (Python Code)

Max Activating Token Index: 385

Text #14

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6357. Min Act: -0.0000

Data Index: 1494624 (Python Code)

Max Activating Token Index: 336

Text #15

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6833. Min Act: -0.0000

Data Index: 1424058 (Python Code)

Max Activating Token Index: 747

Text #16

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6373. Min Act: -0.0000

Data Index: 1562745 (Python Code)

Max Activating Token Index: 475

Text #17

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6523. Min Act: -0.0000

Data Index: 1384181 (Python Code)

Max Activating Token Index: 203

Text #18

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6072. Min Act: -0.0000

Data Index: 1623631 (Python Code)

Max Activating Token Index: 156

Text #19

Max Range: 0.9033. Min Range: -0.9033

Max Act: 0.6236. Min Act: -0.0000

Data Index: 1423711 (Python Code)

Max Activating Token Index: 471