Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 435 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.5510. Min Act: -0.1700

Data Index: 8561098 (Open Web Text)

Max Activating Token Index: 545

Text #1

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.4201. Min Act: -0.1700

Data Index: 7993674 (Open Web Text)

Max Activating Token Index: 943

Text #2

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.3346. Min Act: -0.1700

Data Index: 56189 (Open Web Text)

Max Activating Token Index: 792

Text #3

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.3372. Min Act: -0.1700

Data Index: 6900949 (Open Web Text)

Max Activating Token Index: 705

Text #4

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.2346. Min Act: -0.1700

Data Index: 6063214 (Open Web Text)

Max Activating Token Index: 435

Text #5

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.2294. Min Act: -0.1700

Data Index: 4345551 (Open Web Text)

Max Activating Token Index: 806

Text #6

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.2266. Min Act: -0.1700

Data Index: 6591678 (Open Web Text)

Max Activating Token Index: 850

Text #7

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.2211. Min Act: -0.1700

Data Index: 1415876 (Open Web Text)

Max Activating Token Index: 552

Text #8

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.2207. Min Act: -0.1700

Data Index: 3722417 (Open Web Text)

Max Activating Token Index: 222

Text #9

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1576. Min Act: -0.1700

Data Index: 858126 (Open Web Text)

Max Activating Token Index: 538

Text #10

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1910. Min Act: -0.1700

Data Index: 6713920 (Open Web Text)

Max Activating Token Index: 367

Text #11

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1897. Min Act: -0.1700

Data Index: 7877572 (Open Web Text)

Max Activating Token Index: 236

Text #12

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1582. Min Act: -0.1700

Data Index: 810593 (Open Web Text)

Max Activating Token Index: 690

Text #13

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1089. Min Act: -0.1700

Data Index: 8742175 (Open Web Text)

Max Activating Token Index: 532

Text #14

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1478. Min Act: -0.1700

Data Index: 3171679 (Open Web Text)

Max Activating Token Index: 245

Text #15

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1428. Min Act: -0.1700

Data Index: 2131003 (Open Web Text)

Max Activating Token Index: 407

Text #16

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.0955. Min Act: -0.1700

Data Index: 32968 (Open Web Text)

Max Activating Token Index: 907

Text #17

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.0559. Min Act: -0.1700

Data Index: 3322827 (Open Web Text)

Max Activating Token Index: 619

Text #18

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.1158. Min Act: -0.1700

Data Index: 8364767 (Open Web Text)

Max Activating Token Index: 791

Text #19

Max Range: 4.5510. Min Range: -4.5510

Max Act: 4.0269. Min Act: -0.1700

Data Index: 7698309 (Open Web Text)

Max Activating Token Index: 816