Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2548 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5898. Min Act: -0.1700

Data Index: 675119 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5487. Min Act: -0.1700

Data Index: 2449441 (Open Web Text)

Max Activating Token Index: 20

Text #2

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5284. Min Act: -0.1700

Data Index: 6046259 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5254. Min Act: -0.1700

Data Index: 3209605 (Open Web Text)

Max Activating Token Index: 829

Text #4

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5197. Min Act: -0.1700

Data Index: 8281595 (Open Web Text)

Max Activating Token Index: 41

Text #5

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5086. Min Act: -0.1700

Data Index: 2023217 (Open Web Text)

Max Activating Token Index: 169

Text #6

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4898. Min Act: -0.1700

Data Index: 1364127 (Open Web Text)

Max Activating Token Index: 818

Text #7

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5213. Min Act: -0.1700

Data Index: 5642066 (Open Web Text)

Max Activating Token Index: 321

Text #8

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5037. Min Act: -0.1700

Data Index: 1296886 (Open Web Text)

Max Activating Token Index: 391

Text #9

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4957. Min Act: -0.1700

Data Index: 653546 (Open Web Text)

Max Activating Token Index: 270

Text #10

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4881. Min Act: -0.1700

Data Index: 6347968 (Open Web Text)

Max Activating Token Index: 254

Text #11

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5018. Min Act: -0.1700

Data Index: 5938214 (Open Web Text)

Max Activating Token Index: 926

Text #12

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.5007. Min Act: -0.1700

Data Index: 277713 (Open Web Text)

Max Activating Token Index: 714

Text #13

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4680. Min Act: -0.1700

Data Index: 1960581 (Open Web Text)

Max Activating Token Index: 663

Text #14

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4781. Min Act: -0.1700

Data Index: 1982957 (Open Web Text)

Max Activating Token Index: 51

Text #15

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4612. Min Act: -0.1700

Data Index: 2626928 (Open Web Text)

Max Activating Token Index: 67

Text #16

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4589. Min Act: -0.1700

Data Index: 4089358 (Open Web Text)

Max Activating Token Index: 188

Text #17

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4744. Min Act: -0.1700

Data Index: 6615251 (Open Web Text)

Max Activating Token Index: 735

Text #18

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4628. Min Act: -0.1700

Data Index: 4473495 (Open Web Text)

Max Activating Token Index: 232

Text #19

Max Range: 3.5898. Min Range: -3.5898

Max Act: 3.4510. Min Act: -0.1700

Data Index: 6263577 (Open Web Text)

Max Activating Token Index: 241