Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2129 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.4659. Min Act: -0.1700

Data Index: 8117954 (Open Web Text)

Max Activating Token Index: 997

Text #1

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.3280. Min Act: -0.1700

Data Index: 3335993 (Open Web Text)

Max Activating Token Index: 794

Text #2

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2643. Min Act: -0.1700

Data Index: 6206033 (Open Web Text)

Max Activating Token Index: 868

Text #3

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2569. Min Act: -0.1700

Data Index: 5500529 (Open Web Text)

Max Activating Token Index: 863

Text #4

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2380. Min Act: -0.1700

Data Index: 4972358 (Open Web Text)

Max Activating Token Index: 982

Text #5

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2237. Min Act: -0.1700

Data Index: 5828286 (Open Web Text)

Max Activating Token Index: 536

Text #6

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2247. Min Act: -0.1700

Data Index: 7051921 (Open Web Text)

Max Activating Token Index: 821

Text #7

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2011. Min Act: -0.1700

Data Index: 7014890 (Open Web Text)

Max Activating Token Index: 416

Text #8

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1928. Min Act: -0.1700

Data Index: 2139319 (Open Web Text)

Max Activating Token Index: 53

Text #9

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.2054. Min Act: -0.1700

Data Index: 3140425 (Open Web Text)

Max Activating Token Index: 308

Text #10

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1802. Min Act: -0.1700

Data Index: 6247171 (Open Web Text)

Max Activating Token Index: 1001

Text #11

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1682. Min Act: -0.1700

Data Index: 1075417 (Open Web Text)

Max Activating Token Index: 822

Text #12

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1510. Min Act: -0.1700

Data Index: 728727 (Open Web Text)

Max Activating Token Index: 121

Text #13

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1539. Min Act: -0.1700

Data Index: 1352012 (Open Web Text)

Max Activating Token Index: 510

Text #14

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1393. Min Act: -0.1700

Data Index: 5365000 (Open Web Text)

Max Activating Token Index: 950

Text #15

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1229. Min Act: -0.1700

Data Index: 191360 (Open Web Text)

Max Activating Token Index: 498

Text #16

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1257. Min Act: -0.1700

Data Index: 7375612 (Open Web Text)

Max Activating Token Index: 132

Text #17

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1122. Min Act: -0.1700

Data Index: 5858410 (Open Web Text)

Max Activating Token Index: 295

Text #18

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.0935. Min Act: -0.1700

Data Index: 4949691 (Open Web Text)

Max Activating Token Index: 577

Text #19

Max Range: 3.4659. Min Range: -3.4659

Max Act: 3.1129. Min Act: -0.1700

Data Index: 7157824 (Open Web Text)

Max Activating Token Index: 315