Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2394 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.8346. Min Act: -0.1700

Data Index: 6969337 (Open Web Text)

Max Activating Token Index: 615

Text #1

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.8900. Min Act: -0.1700

Data Index: 423530 (Open Web Text)

Max Activating Token Index: 619

Text #2

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.8209. Min Act: -0.1700

Data Index: 8429357 (Open Web Text)

Max Activating Token Index: 491

Text #3

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.8161. Min Act: -0.1700

Data Index: 6716704 (Open Web Text)

Max Activating Token Index: 292

Text #4

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.8114. Min Act: -0.1700

Data Index: 5104306 (Open Web Text)

Max Activating Token Index: 584

Text #5

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6622. Min Act: -0.1700

Data Index: 205649 (Open Web Text)

Max Activating Token Index: 490

Text #6

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6447. Min Act: -0.1700

Data Index: 8134208 (Open Web Text)

Max Activating Token Index: 118

Text #7

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.7045. Min Act: -0.1700

Data Index: 8353771 (Open Web Text)

Max Activating Token Index: 571

Text #8

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.7585. Min Act: -0.1700

Data Index: 5441492 (Open Web Text)

Max Activating Token Index: 809

Text #9

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.7327. Min Act: -0.1700

Data Index: 4299548 (Open Web Text)

Max Activating Token Index: 436

Text #10

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6585. Min Act: -0.1700

Data Index: 123156 (Open Web Text)

Max Activating Token Index: 516

Text #11

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6337. Min Act: -0.1700

Data Index: 3361582 (Open Web Text)

Max Activating Token Index: 451

Text #12

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6405. Min Act: -0.1700

Data Index: 3133445 (Open Web Text)

Max Activating Token Index: 436

Text #13

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6200. Min Act: -0.1700

Data Index: 2168428 (Open Web Text)

Max Activating Token Index: 690

Text #14

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6757. Min Act: -0.1700

Data Index: 1495098 (Open Web Text)

Max Activating Token Index: 292

Text #15

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6867. Min Act: -0.1700

Data Index: 4477422 (Open Web Text)

Max Activating Token Index: 743

Text #16

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.5434. Min Act: -0.1700

Data Index: 3279677 (Open Web Text)

Max Activating Token Index: 675

Text #17

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.5898. Min Act: -0.1700

Data Index: 1503006 (Open Web Text)

Max Activating Token Index: 517

Text #18

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6082. Min Act: -0.1700

Data Index: 3598290 (Open Web Text)

Max Activating Token Index: 543

Text #19

Max Range: 3.8900. Min Range: -3.8900

Max Act: 3.6138. Min Act: -0.1700

Data Index: 4677865 (Open Web Text)

Max Activating Token Index: 488