Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2813 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0615. Min Act: -0.1700

Data Index: 410202 (Open Web Text)

Max Activating Token Index: 876

Text #1

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0343. Min Act: -0.1700

Data Index: 1579490 (Open Web Text)

Max Activating Token Index: 262

Text #2

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0129. Min Act: -0.1700

Data Index: 2516649 (Open Web Text)

Max Activating Token Index: 820

Text #3

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0434. Min Act: -0.1700

Data Index: 1364167 (Open Web Text)

Max Activating Token Index: 114

Text #4

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0168. Min Act: -0.1700

Data Index: 3613449 (Open Web Text)

Max Activating Token Index: 331

Text #5

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0130. Min Act: -0.1700

Data Index: 6473165 (Open Web Text)

Max Activating Token Index: 473

Text #6

Max Range: 3.0615. Min Range: -3.0615

Max Act: 3.0069. Min Act: -0.1700

Data Index: 7612451 (Open Web Text)

Max Activating Token Index: 706

Text #7

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9583. Min Act: -0.1700

Data Index: 3838383 (Open Web Text)

Max Activating Token Index: 904

Text #8

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9379. Min Act: -0.1700

Data Index: 3865537 (Open Web Text)

Max Activating Token Index: 483

Text #9

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9876. Min Act: -0.1700

Data Index: 6833953 (Open Web Text)

Max Activating Token Index: 863

Text #10

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9514. Min Act: -0.1700

Data Index: 4817488 (Open Web Text)

Max Activating Token Index: 586

Text #11

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9792. Min Act: -0.1700

Data Index: 8106545 (Open Web Text)

Max Activating Token Index: 595

Text #12

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9653. Min Act: -0.1700

Data Index: 7587979 (Open Web Text)

Max Activating Token Index: 1000

Text #13

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9450. Min Act: -0.1700

Data Index: 5579578 (Open Web Text)

Max Activating Token Index: 803

Text #14

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9503. Min Act: -0.1700

Data Index: 6414867 (Open Web Text)

Max Activating Token Index: 879

Text #15

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9651. Min Act: -0.1700

Data Index: 5267839 (Open Web Text)

Max Activating Token Index: 171

Text #16

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9708. Min Act: -0.1700

Data Index: 5254684 (Open Web Text)

Max Activating Token Index: 876

Text #17

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9706. Min Act: -0.1700

Data Index: 7597578 (Open Web Text)

Max Activating Token Index: 873

Text #18

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9300. Min Act: -0.1700

Data Index: 4884209 (Open Web Text)

Max Activating Token Index: 823

Text #19

Max Range: 3.0615. Min Range: -3.0615

Max Act: 2.9302. Min Act: -0.1700

Data Index: 1528508 (Open Web Text)

Max Activating Token Index: 542