Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2991 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.4696. Min Act: -0.1700

Data Index: 5854314 (Open Web Text)

Max Activating Token Index: 357

Text #1

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.2116. Min Act: -0.1700

Data Index: 2495977 (Open Web Text)

Max Activating Token Index: 1013

Text #2

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.2366. Min Act: -0.1700

Data Index: 1047043 (Open Web Text)

Max Activating Token Index: 939

Text #3

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.2082. Min Act: -0.1700

Data Index: 5854315 (Open Web Text)

Max Activating Token Index: 134

Text #4

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.1567. Min Act: -0.1700

Data Index: 6209465 (Open Web Text)

Max Activating Token Index: 971

Text #5

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.1261. Min Act: -0.1700

Data Index: 2676614 (Open Web Text)

Max Activating Token Index: 435

Text #6

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0836. Min Act: -0.1700

Data Index: 4429361 (Open Web Text)

Max Activating Token Index: 172

Text #7

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0751. Min Act: -0.1700

Data Index: 779193 (Open Web Text)

Max Activating Token Index: 514

Text #8

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0239. Min Act: -0.1700

Data Index: 7634077 (Open Web Text)

Max Activating Token Index: 804

Text #9

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0145. Min Act: -0.1700

Data Index: 88895 (Open Web Text)

Max Activating Token Index: 249

Text #10

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0304. Min Act: -0.1700

Data Index: 1702077 (Open Web Text)

Max Activating Token Index: 389

Text #11

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0180. Min Act: -0.1700

Data Index: 3911822 (Open Web Text)

Max Activating Token Index: 523

Text #12

Max Range: 3.4696. Min Range: -3.4696

Max Act: 2.9916. Min Act: -0.1700

Data Index: 1141019 (Open Web Text)

Max Activating Token Index: 911

Text #13

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0110. Min Act: -0.1700

Data Index: 6861788 (Open Web Text)

Max Activating Token Index: 488

Text #14

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0102. Min Act: -0.1700

Data Index: 1351405 (Open Web Text)

Max Activating Token Index: 992

Text #15

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0018. Min Act: -0.1700

Data Index: 1796565 (Open Web Text)

Max Activating Token Index: 598

Text #16

Max Range: 3.4696. Min Range: -3.4696

Max Act: 2.9855. Min Act: -0.1700

Data Index: 4254547 (Open Web Text)

Max Activating Token Index: 429

Text #17

Max Range: 3.4696. Min Range: -3.4696

Max Act: 3.0046. Min Act: -0.1700

Data Index: 7381936 (Open Web Text)

Max Activating Token Index: 523

Text #18

Max Range: 3.4696. Min Range: -3.4696

Max Act: 2.9917. Min Act: -0.1700

Data Index: 5355805 (Open Web Text)

Max Activating Token Index: 232

Text #19

Max Range: 3.4696. Min Range: -3.4696

Max Act: 2.9847. Min Act: -0.1700

Data Index: 7224422 (Open Web Text)

Max Activating Token Index: 100