Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2502 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.8457. Min Act: -0.1700

Data Index: 857848 (Open Web Text)

Max Activating Token Index: 510

Text #1

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.8201. Min Act: -0.1700

Data Index: 2221486 (Open Web Text)

Max Activating Token Index: 242

Text #2

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7941. Min Act: -0.1700

Data Index: 1453960 (Open Web Text)

Max Activating Token Index: 416

Text #3

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7587. Min Act: -0.1700

Data Index: 538540 (Open Web Text)

Max Activating Token Index: 471

Text #4

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7639. Min Act: -0.1700

Data Index: 6252983 (Open Web Text)

Max Activating Token Index: 519

Text #5

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7172. Min Act: -0.1700

Data Index: 5238776 (Open Web Text)

Max Activating Token Index: 549

Text #6

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7220. Min Act: -0.1700

Data Index: 2342438 (Open Web Text)

Max Activating Token Index: 854

Text #7

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6953. Min Act: -0.1700

Data Index: 7848437 (Open Web Text)

Max Activating Token Index: 549

Text #8

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6958. Min Act: -0.1700

Data Index: 6023738 (Open Web Text)

Max Activating Token Index: 982

Text #9

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7340. Min Act: -0.1700

Data Index: 5253096 (Open Web Text)

Max Activating Token Index: 692

Text #10

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7030. Min Act: -0.1700

Data Index: 4001165 (Open Web Text)

Max Activating Token Index: 972

Text #11

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6703. Min Act: -0.1700

Data Index: 3508014 (Open Web Text)

Max Activating Token Index: 638

Text #12

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.7021. Min Act: -0.1700

Data Index: 2581760 (Open Web Text)

Max Activating Token Index: 708

Text #13

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6573. Min Act: -0.1700

Data Index: 234688 (Open Web Text)

Max Activating Token Index: 576

Text #14

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6523. Min Act: -0.1700

Data Index: 8549061 (Open Web Text)

Max Activating Token Index: 906

Text #15

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6596. Min Act: -0.1700

Data Index: 1146266 (Open Web Text)

Max Activating Token Index: 218

Text #16

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6856. Min Act: -0.1700

Data Index: 1813205 (Open Web Text)

Max Activating Token Index: 959

Text #17

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6556. Min Act: -0.1700

Data Index: 539051 (Open Web Text)

Max Activating Token Index: 958

Text #18

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6611. Min Act: -0.1700

Data Index: 8751362 (Open Web Text)

Max Activating Token Index: 149

Text #19

Max Range: 3.8457. Min Range: -3.8457

Max Act: 3.6373. Min Act: -0.1700

Data Index: 8772203 (Open Web Text)

Max Activating Token Index: 948