Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1708 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.7278. Min Act: -0.1700

Data Index: 2977159 (Open Web Text)

Max Activating Token Index: 738

Text #1

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.7095. Min Act: -0.1700

Data Index: 20941 (Open Web Text)

Max Activating Token Index: 740

Text #2

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.6876. Min Act: -0.1700

Data Index: 4133131 (Open Web Text)

Max Activating Token Index: 905

Text #3

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.7092. Min Act: -0.1700

Data Index: 6994197 (Open Web Text)

Max Activating Token Index: 898

Text #4

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.6583. Min Act: -0.1700

Data Index: 1245343 (Open Web Text)

Max Activating Token Index: 189

Text #5

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.6332. Min Act: -0.1700

Data Index: 4231254 (Open Web Text)

Max Activating Token Index: 958

Text #6

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.6251. Min Act: -0.1700

Data Index: 3991529 (Open Web Text)

Max Activating Token Index: 905

Text #7

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.5551. Min Act: -0.1700

Data Index: 2871695 (Open Web Text)

Max Activating Token Index: 667

Text #8

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.5202. Min Act: -0.1700

Data Index: 5390415 (Open Web Text)

Max Activating Token Index: 173

Text #9

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4808. Min Act: -0.1700

Data Index: 6667558 (Open Web Text)

Max Activating Token Index: 950

Text #10

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4748. Min Act: -0.1700

Data Index: 4417440 (Open Web Text)

Max Activating Token Index: 578

Text #11

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4701. Min Act: -0.1700

Data Index: 6381613 (Open Web Text)

Max Activating Token Index: 510

Text #12

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4985. Min Act: -0.1700

Data Index: 2761917 (Open Web Text)

Max Activating Token Index: 309

Text #13

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4681. Min Act: -0.1700

Data Index: 6055130 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4848. Min Act: -0.1700

Data Index: 945716 (Open Web Text)

Max Activating Token Index: 152

Text #15

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4735. Min Act: -0.1700

Data Index: 4928786 (Open Web Text)

Max Activating Token Index: 652

Text #16

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4671. Min Act: -0.1700

Data Index: 1346398 (Open Web Text)

Max Activating Token Index: 186

Text #17

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4606. Min Act: -0.1700

Data Index: 1924632 (Open Web Text)

Max Activating Token Index: 142

Text #18

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4649. Min Act: -0.1700

Data Index: 7887774 (Open Web Text)

Max Activating Token Index: 563

Text #19

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4427. Min Act: -0.1700

Data Index: 4584536 (Open Web Text)

Max Activating Token Index: 278