Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 373 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.4575. Min Act: -0.1700

Data Index: 3453112 (Open Web Text)

Max Activating Token Index: 739

Text #1

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.3551. Min Act: -0.1700

Data Index: 5308980 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.3756. Min Act: -0.1700

Data Index: 4507254 (Open Web Text)

Max Activating Token Index: 22

Text #3

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2482. Min Act: -0.1700

Data Index: 8194768 (Open Web Text)

Max Activating Token Index: 45

Text #4

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.3055. Min Act: -0.1700

Data Index: 8805525 (Open Web Text)

Max Activating Token Index: 169

Text #5

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2554. Min Act: -0.1700

Data Index: 986325 (Open Web Text)

Max Activating Token Index: 783

Text #6

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2474. Min Act: -0.1700

Data Index: 537693 (Open Web Text)

Max Activating Token Index: 72

Text #7

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2371. Min Act: -0.1700

Data Index: 417006 (Open Web Text)

Max Activating Token Index: 711

Text #8

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2286. Min Act: -0.1700

Data Index: 4809654 (Open Web Text)

Max Activating Token Index: 220

Text #9

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1594. Min Act: -0.1700

Data Index: 3553243 (Open Web Text)

Max Activating Token Index: 392

Text #10

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2813. Min Act: -0.1700

Data Index: 5742517 (Open Web Text)

Max Activating Token Index: 792

Text #11

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2240. Min Act: -0.1700

Data Index: 7532255 (Open Web Text)

Max Activating Token Index: 690

Text #12

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2303. Min Act: -0.1700

Data Index: 3191428 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1711. Min Act: -0.1700

Data Index: 3339101 (Open Web Text)

Max Activating Token Index: 122

Text #14

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1882. Min Act: -0.1700

Data Index: 4769902 (Open Web Text)

Max Activating Token Index: 521

Text #15

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.2388. Min Act: -0.1700

Data Index: 3432024 (Open Web Text)

Max Activating Token Index: 716

Text #16

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.0985. Min Act: -0.1700

Data Index: 7301520 (Open Web Text)

Max Activating Token Index: 883

Text #17

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1778. Min Act: -0.1700

Data Index: 2736470 (Open Web Text)

Max Activating Token Index: 505

Text #18

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1418. Min Act: -0.1700

Data Index: 1268081 (Open Web Text)

Max Activating Token Index: 152

Text #19

Max Range: 5.4575. Min Range: -5.4575

Max Act: 5.1517. Min Act: -0.1700

Data Index: 6151681 (Open Web Text)

Max Activating Token Index: 1000