Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1764 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.2050. Min Act: -0.1700

Data Index: 7865661 (Open Web Text)

Max Activating Token Index: 703

Text #1

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1631. Min Act: -0.1700

Data Index: 2025257 (Open Web Text)

Max Activating Token Index: 645

Text #2

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1765. Min Act: -0.1700

Data Index: 2025258 (Open Web Text)

Max Activating Token Index: 406

Text #3

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1148. Min Act: -0.1700

Data Index: 3099589 (Open Web Text)

Max Activating Token Index: 309

Text #4

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1108. Min Act: -0.1700

Data Index: 5569578 (Open Web Text)

Max Activating Token Index: 185

Text #5

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1318. Min Act: -0.1700

Data Index: 7660385 (Open Web Text)

Max Activating Token Index: 304

Text #6

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.1064. Min Act: -0.1700

Data Index: 307124 (Open Web Text)

Max Activating Token Index: 715

Text #7

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0955. Min Act: -0.1700

Data Index: 6820440 (Open Web Text)

Max Activating Token Index: 259

Text #8

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0827. Min Act: -0.1700

Data Index: 4038710 (Open Web Text)

Max Activating Token Index: 650

Text #9

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0699. Min Act: -0.1700

Data Index: 2893470 (Open Web Text)

Max Activating Token Index: 13

Text #10

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0779. Min Act: -0.1700

Data Index: 6367520 (Open Web Text)

Max Activating Token Index: 516

Text #11

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0886. Min Act: -0.1700

Data Index: 1108817 (Open Web Text)

Max Activating Token Index: 749

Text #12

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0701. Min Act: -0.1700

Data Index: 2994773 (Open Web Text)

Max Activating Token Index: 704

Text #13

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0735. Min Act: -0.1700

Data Index: 5981919 (Open Web Text)

Max Activating Token Index: 245

Text #14

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0700. Min Act: -0.1700

Data Index: 5161030 (Open Web Text)

Max Activating Token Index: 291

Text #15

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0372. Min Act: -0.1700

Data Index: 4820749 (Open Web Text)

Max Activating Token Index: 617

Text #16

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0839. Min Act: -0.1700

Data Index: 7676698 (Open Web Text)

Max Activating Token Index: 760

Text #17

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0309. Min Act: -0.1700

Data Index: 6444388 (Open Web Text)

Max Activating Token Index: 999

Text #18

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0295. Min Act: -0.1700

Data Index: 3757189 (Open Web Text)

Max Activating Token Index: 762

Text #19

Max Range: 5.2050. Min Range: -5.2050

Max Act: 5.0598. Min Act: -0.1700

Data Index: 1483104 (Open Web Text)

Max Activating Token Index: 992