Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 193 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.3397. Min Act: -0.0000

Data Index: 313357 (The Pile)

Max Activating Token Index: 900

Text #1

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0635. Min Act: -0.0000

Data Index: 1214622 (The Pile)

Max Activating Token Index: 237

Text #2

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0536. Min Act: -0.0000

Data Index: 929959 (The Pile)

Max Activating Token Index: 373

Text #3

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0452. Min Act: -0.0000

Data Index: 1284864 (The Pile)

Max Activating Token Index: 411

Text #4

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0614. Min Act: -0.0000

Data Index: 1099786 (The Pile)

Max Activating Token Index: 673

Text #5

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0188. Min Act: -0.0000

Data Index: 1625183 (The Pile)

Max Activating Token Index: 253

Text #6

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0064. Min Act: -0.0000

Data Index: 1145830 (The Pile)

Max Activating Token Index: 444

Text #7

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9921. Min Act: -0.0000

Data Index: 593362 (The Pile)

Max Activating Token Index: 421

Text #8

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0006. Min Act: -0.0000

Data Index: 493163 (The Pile)

Max Activating Token Index: 749

Text #9

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9572. Min Act: -0.0000

Data Index: 1986667 (The Pile)

Max Activating Token Index: 520

Text #10

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9810. Min Act: -0.0000

Data Index: 1588928 (The Pile)

Max Activating Token Index: 71

Text #11

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0434. Min Act: -0.0000

Data Index: 612211 (The Pile)

Max Activating Token Index: 815

Text #12

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9806. Min Act: -0.0000

Data Index: 958601 (The Pile)

Max Activating Token Index: 1020

Text #13

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9703. Min Act: -0.0000

Data Index: 44407 (The Pile)

Max Activating Token Index: 425

Text #14

Max Range: 1.3397. Min Range: -1.3397

Max Act: 1.0072. Min Act: -0.0000

Data Index: 906581 (The Pile)

Max Activating Token Index: 37

Text #15

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9322. Min Act: -0.0000

Data Index: 887041 (The Pile)

Max Activating Token Index: 507

Text #16

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9589. Min Act: -0.0000

Data Index: 1686198 (The Pile)

Max Activating Token Index: 286

Text #17

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9307. Min Act: -0.0000

Data Index: 1414157 (The Pile)

Max Activating Token Index: 502

Text #18

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9835. Min Act: -0.0000

Data Index: 1698273 (The Pile)

Max Activating Token Index: 605

Text #19

Max Range: 1.3397. Min Range: -1.3397

Max Act: 0.9830. Min Act: -0.0000

Data Index: 1600722 (The Pile)

Max Activating Token Index: 223