Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 789 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0842. Min Range: -3.0842

Max Act: 3.0842. Min Act: -0.1700

Data Index: 5817541 (Open Web Text)

Max Activating Token Index: 573

Text #1

Max Range: 3.0842. Min Range: -3.0842

Max Act: 3.0367. Min Act: -0.1700

Data Index: 8312036 (Open Web Text)

Max Activating Token Index: 40

Text #2

Max Range: 3.0842. Min Range: -3.0842

Max Act: 3.0280. Min Act: -0.1700

Data Index: 7305022 (Open Web Text)

Max Activating Token Index: 663

Text #3

Max Range: 3.0842. Min Range: -3.0842

Max Act: 3.0240. Min Act: -0.1700

Data Index: 7926346 (Open Web Text)

Max Activating Token Index: 422

Text #4

Max Range: 3.0842. Min Range: -3.0842

Max Act: 3.0004. Min Act: -0.1700

Data Index: 7795150 (Open Web Text)

Max Activating Token Index: 250

Text #5

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9890. Min Act: -0.1700

Data Index: 7455990 (Open Web Text)

Max Activating Token Index: 664

Text #6

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9980. Min Act: -0.1700

Data Index: 4791071 (Open Web Text)

Max Activating Token Index: 577

Text #7

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9593. Min Act: -0.1700

Data Index: 1348725 (Open Web Text)

Max Activating Token Index: 683

Text #8

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9692. Min Act: -0.1700

Data Index: 2456678 (Open Web Text)

Max Activating Token Index: 461

Text #9

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9699. Min Act: -0.1700

Data Index: 8571420 (Open Web Text)

Max Activating Token Index: 581

Text #10

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9472. Min Act: -0.1700

Data Index: 4030696 (Open Web Text)

Max Activating Token Index: 953

Text #11

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9394. Min Act: -0.1700

Data Index: 598113 (Open Web Text)

Max Activating Token Index: 886

Text #12

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9411. Min Act: -0.1700

Data Index: 6278580 (Open Web Text)

Max Activating Token Index: 894

Text #13

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9388. Min Act: -0.1700

Data Index: 4878256 (Open Web Text)

Max Activating Token Index: 355

Text #14

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9375. Min Act: -0.1700

Data Index: 23520 (Open Web Text)

Max Activating Token Index: 296

Text #15

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9346. Min Act: -0.1700

Data Index: 1352030 (Open Web Text)

Max Activating Token Index: 632

Text #16

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9268. Min Act: -0.1700

Data Index: 4918429 (Open Web Text)

Max Activating Token Index: 202

Text #17

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9477. Min Act: -0.1700

Data Index: 1258334 (Open Web Text)

Max Activating Token Index: 154

Text #18

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9399. Min Act: -0.1700

Data Index: 951109 (Open Web Text)

Max Activating Token Index: 282

Text #19

Max Range: 3.0842. Min Range: -3.0842

Max Act: 2.9231. Min Act: -0.1700

Data Index: 741471 (Open Web Text)

Max Activating Token Index: 614