Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 143 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.5351. Min Act: -0.0000

Data Index: 294149 (The Pile)

Max Activating Token Index: 569

Text #1

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4759. Min Act: -0.0000

Data Index: 345566 (The Pile)

Max Activating Token Index: 894

Text #2

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4912. Min Act: -0.0000

Data Index: 1152629 (The Pile)

Max Activating Token Index: 868

Text #3

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4698. Min Act: -0.0000

Data Index: 1139702 (The Pile)

Max Activating Token Index: 126

Text #4

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4820. Min Act: -0.0000

Data Index: 1632745 (The Pile)

Max Activating Token Index: 139

Text #5

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4766. Min Act: -0.0000

Data Index: 1712974 (The Pile)

Max Activating Token Index: 245

Text #6

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4410. Min Act: -0.0000

Data Index: 59805 (The Pile)

Max Activating Token Index: 473

Text #7

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4462. Min Act: -0.0000

Data Index: 1769675 (The Pile)

Max Activating Token Index: 844

Text #8

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4459. Min Act: -0.0000

Data Index: 1544657 (The Pile)

Max Activating Token Index: 393

Text #9

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4584. Min Act: 0.0000

Data Index: 1852469 (The Pile)

Max Activating Token Index: 622

Text #10

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4309. Min Act: -0.0000

Data Index: 1282072 (The Pile)

Max Activating Token Index: 443

Text #11

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4342. Min Act: -0.0000

Data Index: 338062 (The Pile)

Max Activating Token Index: 538

Text #12

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4193. Min Act: 0.0000

Data Index: 1453159 (The Pile)

Max Activating Token Index: 723

Text #13

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4412. Min Act: 0.0000

Data Index: 1913443 (The Pile)

Max Activating Token Index: 283

Text #14

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4222. Min Act: -0.0000

Data Index: 1931070 (The Pile)

Max Activating Token Index: 998

Text #15

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4145. Min Act: -0.0000

Data Index: 1728058 (The Pile)

Max Activating Token Index: 894

Text #16

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4397. Min Act: -0.0000

Data Index: 574626 (The Pile)

Max Activating Token Index: 939

Text #17

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.3854. Min Act: -0.0000

Data Index: 1665519 (The Pile)

Max Activating Token Index: 307

Text #18

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.4051. Min Act: 0.0000

Data Index: 1634727 (The Pile)

Max Activating Token Index: 358

Text #19

Max Range: 0.5351. Min Range: -0.5351

Max Act: 0.3845. Min Act: -0.0000

Data Index: 885081 (The Pile)

Max Activating Token Index: 436