Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2030 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.1807. Min Act: -0.1700

Data Index: 4855036 (Open Web Text)

Max Activating Token Index: 980

Text #1

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.1384. Min Act: -0.1700

Data Index: 5421433 (Open Web Text)

Max Activating Token Index: 731

Text #2

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.1435. Min Act: -0.1700

Data Index: 2778732 (Open Web Text)

Max Activating Token Index: 304

Text #3

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0363. Min Act: -0.1700

Data Index: 4703267 (Open Web Text)

Max Activating Token Index: 403

Text #4

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0351. Min Act: -0.1700

Data Index: 2538408 (Open Web Text)

Max Activating Token Index: 685

Text #5

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0317. Min Act: -0.1700

Data Index: 1284406 (Open Web Text)

Max Activating Token Index: 773

Text #6

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0294. Min Act: -0.1700

Data Index: 6865377 (Open Web Text)

Max Activating Token Index: 452

Text #7

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0728. Min Act: -0.1700

Data Index: 3668471 (Open Web Text)

Max Activating Token Index: 90

Text #8

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0836. Min Act: -0.1700

Data Index: 717253 (Open Web Text)

Max Activating Token Index: 922

Text #9

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0538. Min Act: -0.1700

Data Index: 6191671 (Open Web Text)

Max Activating Token Index: 991

Text #10

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0511. Min Act: -0.1700

Data Index: 5606666 (Open Web Text)

Max Activating Token Index: 755

Text #11

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0284. Min Act: -0.1700

Data Index: 2808928 (Open Web Text)

Max Activating Token Index: 198

Text #12

Max Range: 4.1807. Min Range: -4.1807

Max Act: 3.9927. Min Act: -0.1700

Data Index: 5500063 (Open Web Text)

Max Activating Token Index: 920

Text #13

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0558. Min Act: -0.1700

Data Index: 6137556 (Open Web Text)

Max Activating Token Index: 243

Text #14

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0409. Min Act: -0.1700

Data Index: 149781 (Open Web Text)

Max Activating Token Index: 783

Text #15

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0508. Min Act: -0.1700

Data Index: 1776371 (Open Web Text)

Max Activating Token Index: 288

Text #16

Max Range: 4.1807. Min Range: -4.1807

Max Act: 3.9839. Min Act: -0.1700

Data Index: 1133571 (Open Web Text)

Max Activating Token Index: 69

Text #17

Max Range: 4.1807. Min Range: -4.1807

Max Act: 3.9983. Min Act: -0.1700

Data Index: 5484496 (Open Web Text)

Max Activating Token Index: 931

Text #18

Max Range: 4.1807. Min Range: -4.1807

Max Act: 4.0437. Min Act: -0.1700

Data Index: 7373359 (Open Web Text)

Max Activating Token Index: 926

Text #19

Max Range: 4.1807. Min Range: -4.1807

Max Act: 3.9853. Min Act: -0.1700

Data Index: 3725714 (Open Web Text)

Max Activating Token Index: 662