Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 102 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.4331. Min Act: -0.1700

Data Index: 5245266 (Open Web Text)

Max Activating Token Index: 421

Text #1

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.2815. Min Act: -0.1700

Data Index: 5228956 (Open Web Text)

Max Activating Token Index: 262

Text #2

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.2383. Min Act: -0.1700

Data Index: 1385543 (Open Web Text)

Max Activating Token Index: 587

Text #3

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.2264. Min Act: -0.1700

Data Index: 3577383 (Open Web Text)

Max Activating Token Index: 710

Text #4

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.2460. Min Act: -0.1700

Data Index: 2105705 (Open Web Text)

Max Activating Token Index: 244

Text #5

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.2226. Min Act: -0.1700

Data Index: 6324891 (Open Web Text)

Max Activating Token Index: 781

Text #6

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1765. Min Act: -0.1700

Data Index: 1249794 (Open Web Text)

Max Activating Token Index: 601

Text #7

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1958. Min Act: -0.1700

Data Index: 5819290 (Open Web Text)

Max Activating Token Index: 242

Text #8

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1696. Min Act: -0.1700

Data Index: 5377613 (Open Web Text)

Max Activating Token Index: 791

Text #9

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1718. Min Act: -0.1700

Data Index: 444406 (Open Web Text)

Max Activating Token Index: 613

Text #10

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1933. Min Act: -0.1700

Data Index: 7639461 (Open Web Text)

Max Activating Token Index: 967

Text #11

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1669. Min Act: -0.1700

Data Index: 657719 (Open Web Text)

Max Activating Token Index: 247

Text #12

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1882. Min Act: -0.1700

Data Index: 6577424 (Open Web Text)

Max Activating Token Index: 850

Text #13

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1431. Min Act: -0.1700

Data Index: 1371086 (Open Web Text)

Max Activating Token Index: 768

Text #14

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1980. Min Act: -0.1700

Data Index: 5721089 (Open Web Text)

Max Activating Token Index: 696

Text #15

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1582. Min Act: -0.1700

Data Index: 8034684 (Open Web Text)

Max Activating Token Index: 199

Text #16

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1637. Min Act: -0.1700

Data Index: 3909091 (Open Web Text)

Max Activating Token Index: 107

Text #17

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1744. Min Act: -0.1700

Data Index: 7823474 (Open Web Text)

Max Activating Token Index: 329

Text #18

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1504. Min Act: -0.1700

Data Index: 2132381 (Open Web Text)

Max Activating Token Index: 731

Text #19

Max Range: 3.4331. Min Range: -3.4331

Max Act: 3.1582. Min Act: -0.1700

Data Index: 7267170 (Open Web Text)

Max Activating Token Index: 24