Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2891 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7955. Min Act: -0.1700

Data Index: 5370483 (Open Web Text)

Max Activating Token Index: 601

Text #1

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7656. Min Act: -0.1700

Data Index: 7642683 (Open Web Text)

Max Activating Token Index: 844

Text #2

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.8001. Min Act: -0.1700

Data Index: 8080439 (Open Web Text)

Max Activating Token Index: 1009

Text #3

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7814. Min Act: -0.1700

Data Index: 3791670 (Open Web Text)

Max Activating Token Index: 230

Text #4

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7795. Min Act: -0.1700

Data Index: 4048638 (Open Web Text)

Max Activating Token Index: 790

Text #5

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7811. Min Act: -0.1700

Data Index: 3881526 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7085. Min Act: -0.1700

Data Index: 4031935 (Open Web Text)

Max Activating Token Index: 947

Text #7

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.7601. Min Act: -0.1700

Data Index: 1233735 (Open Web Text)

Max Activating Token Index: 220

Text #8

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6397. Min Act: -0.1700

Data Index: 1283974 (Open Web Text)

Max Activating Token Index: 401

Text #9

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6912. Min Act: -0.1700

Data Index: 2428092 (Open Web Text)

Max Activating Token Index: 76

Text #10

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6216. Min Act: -0.1700

Data Index: 644302 (Open Web Text)

Max Activating Token Index: 676

Text #11

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6980. Min Act: -0.1700

Data Index: 3285848 (Open Web Text)

Max Activating Token Index: 249

Text #12

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6689. Min Act: -0.1700

Data Index: 3370019 (Open Web Text)

Max Activating Token Index: 151

Text #13

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6171. Min Act: -0.1700

Data Index: 1734302 (Open Web Text)

Max Activating Token Index: 97

Text #14

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6296. Min Act: -0.1700

Data Index: 1747437 (Open Web Text)

Max Activating Token Index: 437

Text #15

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6079. Min Act: -0.1700

Data Index: 1137115 (Open Web Text)

Max Activating Token Index: 993

Text #16

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6222. Min Act: -0.1700

Data Index: 405246 (Open Web Text)

Max Activating Token Index: 548

Text #17

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.6233. Min Act: -0.1700

Data Index: 3811117 (Open Web Text)

Max Activating Token Index: 773

Text #18

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.5865. Min Act: -0.1700

Data Index: 5587399 (Open Web Text)

Max Activating Token Index: 471

Text #19

Max Range: 4.8001. Min Range: -4.8001

Max Act: 4.5398. Min Act: -0.1700

Data Index: 3727604 (Open Web Text)

Max Activating Token Index: 661