Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2736 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.3372. Min Act: -0.1700

Data Index: 2348085 (Open Web Text)

Max Activating Token Index: 37

Text #1

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.3129. Min Act: -0.1700

Data Index: 1360524 (Open Web Text)

Max Activating Token Index: 232

Text #2

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.3260. Min Act: -0.1700

Data Index: 8478636 (Open Web Text)

Max Activating Token Index: 471

Text #3

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2945. Min Act: -0.1700

Data Index: 1407267 (Open Web Text)

Max Activating Token Index: 514

Text #4

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2502. Min Act: -0.1700

Data Index: 1979133 (Open Web Text)

Max Activating Token Index: 603

Text #5

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2328. Min Act: -0.1700

Data Index: 1692530 (Open Web Text)

Max Activating Token Index: 421

Text #6

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2487. Min Act: -0.1700

Data Index: 898312 (Open Web Text)

Max Activating Token Index: 826

Text #7

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2652. Min Act: -0.1700

Data Index: 626036 (Open Web Text)

Max Activating Token Index: 927

Text #8

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2311. Min Act: -0.1700

Data Index: 429463 (Open Web Text)

Max Activating Token Index: 164

Text #9

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2327. Min Act: -0.1700

Data Index: 7226049 (Open Web Text)

Max Activating Token Index: 268

Text #10

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2151. Min Act: -0.1700

Data Index: 2830904 (Open Web Text)

Max Activating Token Index: 102

Text #11

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2093. Min Act: -0.1700

Data Index: 2952865 (Open Web Text)

Max Activating Token Index: 675

Text #12

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.2271. Min Act: -0.1700

Data Index: 3122889 (Open Web Text)

Max Activating Token Index: 467

Text #13

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1920. Min Act: -0.1700

Data Index: 8528193 (Open Web Text)

Max Activating Token Index: 363

Text #14

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1981. Min Act: -0.1700

Data Index: 6899456 (Open Web Text)

Max Activating Token Index: 127

Text #15

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1748. Min Act: -0.1700

Data Index: 5532001 (Open Web Text)

Max Activating Token Index: 234

Text #16

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1740. Min Act: -0.1700

Data Index: 203902 (Open Web Text)

Max Activating Token Index: 346

Text #17

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1764. Min Act: -0.1700

Data Index: 4088459 (Open Web Text)

Max Activating Token Index: 948

Text #18

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1678. Min Act: -0.1700

Data Index: 2886949 (Open Web Text)

Max Activating Token Index: 144

Text #19

Max Range: 3.3372. Min Range: -3.3372

Max Act: 3.1536. Min Act: -0.1700

Data Index: 5293595 (Open Web Text)

Max Activating Token Index: 285