Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 252 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1908. Min Act: -0.1700

Data Index: 1687646 (Open Web Text)

Max Activating Token Index: 811

Text #1

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1754. Min Act: -0.1700

Data Index: 4446215 (Open Web Text)

Max Activating Token Index: 702

Text #2

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1962. Min Act: -0.1700

Data Index: 2005170 (Open Web Text)

Max Activating Token Index: 664

Text #3

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1506. Min Act: -0.1700

Data Index: 8608306 (Open Web Text)

Max Activating Token Index: 150

Text #4

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1631. Min Act: -0.1700

Data Index: 4107962 (Open Web Text)

Max Activating Token Index: 477

Text #5

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1219. Min Act: -0.1700

Data Index: 702549 (Open Web Text)

Max Activating Token Index: 663

Text #6

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1459. Min Act: -0.1700

Data Index: 2314241 (Open Web Text)

Max Activating Token Index: 685

Text #7

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1476. Min Act: -0.1700

Data Index: 4107967 (Open Web Text)

Max Activating Token Index: 830

Text #8

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.1180. Min Act: -0.1700

Data Index: 4681332 (Open Web Text)

Max Activating Token Index: 465

Text #9

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0825. Min Act: -0.1700

Data Index: 2016492 (Open Web Text)

Max Activating Token Index: 667

Text #10

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0648. Min Act: -0.1700

Data Index: 516366 (Open Web Text)

Max Activating Token Index: 446

Text #11

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0667. Min Act: -0.1700

Data Index: 914448 (Open Web Text)

Max Activating Token Index: 20

Text #12

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0539. Min Act: -0.1700

Data Index: 2941686 (Open Web Text)

Max Activating Token Index: 741

Text #13

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0677. Min Act: -0.1700

Data Index: 2968738 (Open Web Text)

Max Activating Token Index: 831

Text #14

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0446. Min Act: -0.1700

Data Index: 1029557 (Open Web Text)

Max Activating Token Index: 88

Text #15

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0402. Min Act: -0.1700

Data Index: 6717052 (Open Web Text)

Max Activating Token Index: 802

Text #16

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0423. Min Act: -0.1700

Data Index: 195086 (Open Web Text)

Max Activating Token Index: 124

Text #17

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0215. Min Act: -0.1700

Data Index: 7424991 (Open Web Text)

Max Activating Token Index: 579

Text #18

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0391. Min Act: -0.1700

Data Index: 4125450 (Open Web Text)

Max Activating Token Index: 664

Text #19

Max Range: 3.1962. Min Range: -3.1962

Max Act: 3.0195. Min Act: -0.1700

Data Index: 5860355 (Open Web Text)

Max Activating Token Index: 1001