Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1229 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.3288. Min Act: -0.1700

Data Index: 7953802 (Open Web Text)

Max Activating Token Index: 431

Text #1

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2538. Min Act: -0.1700

Data Index: 2768045 (Open Web Text)

Max Activating Token Index: 774

Text #2

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2234. Min Act: -0.1700

Data Index: 1843174 (Open Web Text)

Max Activating Token Index: 339

Text #3

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2390. Min Act: -0.1700

Data Index: 4049913 (Open Web Text)

Max Activating Token Index: 742

Text #4

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2056. Min Act: -0.1697

Data Index: 8024523 (Open Web Text)

Max Activating Token Index: 937

Text #5

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2066. Min Act: -0.1700

Data Index: 6179639 (Open Web Text)

Max Activating Token Index: 978

Text #6

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2329. Min Act: -0.1700

Data Index: 6137545 (Open Web Text)

Max Activating Token Index: 775

Text #7

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2121. Min Act: -0.1700

Data Index: 4198275 (Open Web Text)

Max Activating Token Index: 161

Text #8

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.2155. Min Act: -0.1700

Data Index: 5283869 (Open Web Text)

Max Activating Token Index: 857

Text #9

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1680. Min Act: -0.1700

Data Index: 6968477 (Open Web Text)

Max Activating Token Index: 954

Text #10

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1572. Min Act: -0.1700

Data Index: 8257038 (Open Web Text)

Max Activating Token Index: 76

Text #11

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1751. Min Act: -0.1700

Data Index: 5843015 (Open Web Text)

Max Activating Token Index: 849

Text #12

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1484. Min Act: -0.1700

Data Index: 5241746 (Open Web Text)

Max Activating Token Index: 876

Text #13

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1771. Min Act: -0.1700

Data Index: 3501466 (Open Web Text)

Max Activating Token Index: 625

Text #14

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1712. Min Act: -0.1700

Data Index: 2433022 (Open Web Text)

Max Activating Token Index: 473

Text #15

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.0889. Min Act: -0.1700

Data Index: 7269076 (Open Web Text)

Max Activating Token Index: 875

Text #16

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1496. Min Act: -0.1700

Data Index: 6442500 (Open Web Text)

Max Activating Token Index: 605

Text #17

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1485. Min Act: -0.1700

Data Index: 7275515 (Open Web Text)

Max Activating Token Index: 140

Text #18

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1410. Min Act: -0.1694

Data Index: 7818587 (Open Web Text)

Max Activating Token Index: 655

Text #19

Max Range: 3.3288. Min Range: -3.3288

Max Act: 3.1412. Min Act: -0.1700

Data Index: 6751189 (Open Web Text)

Max Activating Token Index: 964