Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4956 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.1650. Min Act: -0.0000

Data Index: 1277873 (The Pile)

Max Activating Token Index: 345

Text #1

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.2370. Min Act: -0.0000

Data Index: 297346 (The Pile)

Max Activating Token Index: 382

Text #2

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.2487. Min Act: -0.0000

Data Index: 415167 (The Pile)

Max Activating Token Index: 326

Text #3

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.0961. Min Act: -0.0000

Data Index: 72579 (The Pile)

Max Activating Token Index: 429

Text #4

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.1555. Min Act: -0.0000

Data Index: 916912 (The Pile)

Max Activating Token Index: 1018

Text #5

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.0961. Min Act: 0.0000

Data Index: 498091 (The Pile)

Max Activating Token Index: 178

Text #6

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9915. Min Act: 0.0000

Data Index: 1528494 (The Pile)

Max Activating Token Index: 317

Text #7

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.0470. Min Act: -0.0000

Data Index: 1354413 (The Pile)

Max Activating Token Index: 578

Text #8

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9955. Min Act: 0.0000

Data Index: 1274872 (The Pile)

Max Activating Token Index: 664

Text #9

Max Range: 1.2487. Min Range: -1.2487

Max Act: 1.0463. Min Act: 0.0000

Data Index: 477457 (The Pile)

Max Activating Token Index: 801

Text #10

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9776. Min Act: -0.0000

Data Index: 499954 (The Pile)

Max Activating Token Index: 75

Text #11

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9460. Min Act: -0.0000

Data Index: 1136877 (The Pile)

Max Activating Token Index: 1021

Text #12

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8873. Min Act: -0.0000

Data Index: 1422407 (The Pile)

Max Activating Token Index: 914

Text #13

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8809. Min Act: -0.0000

Data Index: 449230 (The Pile)

Max Activating Token Index: 579

Text #14

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8912. Min Act: -0.0000

Data Index: 1624050 (The Pile)

Max Activating Token Index: 242

Text #15

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9613. Min Act: -0.0000

Data Index: 1012618 (The Pile)

Max Activating Token Index: 931

Text #16

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8321. Min Act: -0.0000

Data Index: 768092 (The Pile)

Max Activating Token Index: 421

Text #17

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8803. Min Act: 0.0000

Data Index: 1038380 (The Pile)

Max Activating Token Index: 717

Text #18

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.9014. Min Act: -0.0000

Data Index: 140562 (The Pile)

Max Activating Token Index: 840

Text #19

Max Range: 1.2487. Min Range: -1.2487

Max Act: 0.8614. Min Act: -0.0000

Data Index: 1320161 (The Pile)

Max Activating Token Index: 18