Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 594 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.6700. Min Act: -0.1700

Data Index: 4835408 (Open Web Text)

Max Activating Token Index: 847

Text #1

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.6929. Min Act: -0.1700

Data Index: 5197278 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.6287. Min Act: -0.1700

Data Index: 6405704 (Open Web Text)

Max Activating Token Index: 603

Text #3

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.6431. Min Act: -0.1700

Data Index: 3488649 (Open Web Text)

Max Activating Token Index: 243

Text #4

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.5791. Min Act: -0.1700

Data Index: 6542806 (Open Web Text)

Max Activating Token Index: 755

Text #5

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.5926. Min Act: -0.1700

Data Index: 7010195 (Open Web Text)

Max Activating Token Index: 25

Text #6

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.5494. Min Act: -0.1700

Data Index: 7258673 (Open Web Text)

Max Activating Token Index: 941

Text #7

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4913. Min Act: -0.1700

Data Index: 3078079 (Open Web Text)

Max Activating Token Index: 953

Text #8

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.5318. Min Act: -0.1700

Data Index: 5214892 (Open Web Text)

Max Activating Token Index: 284

Text #9

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4993. Min Act: -0.1700

Data Index: 7179328 (Open Web Text)

Max Activating Token Index: 621

Text #10

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4366. Min Act: -0.1700

Data Index: 4248700 (Open Web Text)

Max Activating Token Index: 147

Text #11

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4394. Min Act: -0.1700

Data Index: 38977 (Open Web Text)

Max Activating Token Index: 813

Text #12

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4698. Min Act: -0.1700

Data Index: 5627622 (Open Web Text)

Max Activating Token Index: 687

Text #13

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4377. Min Act: -0.1700

Data Index: 6065690 (Open Web Text)

Max Activating Token Index: 592

Text #14

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4484. Min Act: -0.1700

Data Index: 2606513 (Open Web Text)

Max Activating Token Index: 95

Text #15

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4173. Min Act: -0.1700

Data Index: 5487348 (Open Web Text)

Max Activating Token Index: 181

Text #16

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4471. Min Act: -0.1700

Data Index: 5034914 (Open Web Text)

Max Activating Token Index: 201

Text #17

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4474. Min Act: -0.1700

Data Index: 4343797 (Open Web Text)

Max Activating Token Index: 398

Text #18

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4097. Min Act: -0.1700

Data Index: 2066556 (Open Web Text)

Max Activating Token Index: 420

Text #19

Max Range: 3.6929. Min Range: -3.6929

Max Act: 3.4206. Min Act: -0.1700

Data Index: 6336419 (Open Web Text)

Max Activating Token Index: 761