Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 418 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.0928. Min Range: -5.0928

Max Act: 5.0928. Min Act: -0.0000

Data Index: 1132161 (The Pile)

Max Activating Token Index: 664

Text #1

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.9234. Min Act: -0.0000

Data Index: 1496788 (The Pile)

Max Activating Token Index: 668

Text #2

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.7061. Min Act: -0.0000

Data Index: 1976531 (The Pile)

Max Activating Token Index: 504

Text #3

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5648. Min Act: -0.0000

Data Index: 1579473 (The Pile)

Max Activating Token Index: 728

Text #4

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5881. Min Act: -0.0000

Data Index: 140266 (The Pile)

Max Activating Token Index: 880

Text #5

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5604. Min Act: -0.0000

Data Index: 144609 (The Pile)

Max Activating Token Index: 893

Text #6

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5119. Min Act: -0.0000

Data Index: 629356 (The Pile)

Max Activating Token Index: 624

Text #7

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.6388. Min Act: -0.0000

Data Index: 1503000 (The Pile)

Max Activating Token Index: 988

Text #8

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.4209. Min Act: -0.0000

Data Index: 782136 (The Pile)

Max Activating Token Index: 986

Text #9

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.4077. Min Act: -0.0000

Data Index: 685953 (The Pile)

Max Activating Token Index: 989

Text #10

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.7291. Min Act: -0.0000

Data Index: 1639644 (The Pile)

Max Activating Token Index: 716

Text #11

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5526. Min Act: -0.0000

Data Index: 726546 (The Pile)

Max Activating Token Index: 334

Text #12

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.6909. Min Act: -0.0000

Data Index: 1129171 (The Pile)

Max Activating Token Index: 493

Text #13

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.4748. Min Act: -0.0000

Data Index: 1040536 (The Pile)

Max Activating Token Index: 517

Text #14

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.3095. Min Act: -0.0000

Data Index: 287060 (The Pile)

Max Activating Token Index: 387

Text #15

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.6535. Min Act: -0.0000

Data Index: 1866389 (The Pile)

Max Activating Token Index: 552

Text #16

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.4232. Min Act: -0.0000

Data Index: 1960474 (The Pile)

Max Activating Token Index: 1016

Text #17

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.2593. Min Act: -0.0000

Data Index: 605216 (The Pile)

Max Activating Token Index: 1021

Text #18

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.5661. Min Act: -0.0000

Data Index: 1801416 (The Pile)

Max Activating Token Index: 354

Text #19

Max Range: 5.0928. Min Range: -5.0928

Max Act: 4.3337. Min Act: -0.0000

Data Index: 838265 (The Pile)

Max Activating Token Index: 291