Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2416 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7849. Min Act: -0.1697

Data Index: 1734806 (Open Web Text)

Max Activating Token Index: 460

Text #1

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7660. Min Act: -0.1699

Data Index: 1501541 (Open Web Text)

Max Activating Token Index: 517

Text #2

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7712. Min Act: -0.1700

Data Index: 6408584 (Open Web Text)

Max Activating Token Index: 534

Text #3

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7567. Min Act: -0.1700

Data Index: 4317884 (Open Web Text)

Max Activating Token Index: 774

Text #4

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7582. Min Act: -0.1700

Data Index: 4240391 (Open Web Text)

Max Activating Token Index: 41

Text #5

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7542. Min Act: -0.1700

Data Index: 2809869 (Open Web Text)

Max Activating Token Index: 178

Text #6

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7581. Min Act: -0.1700

Data Index: 1532116 (Open Web Text)

Max Activating Token Index: 756

Text #7

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7536. Min Act: -0.1700

Data Index: 1734808 (Open Web Text)

Max Activating Token Index: 835

Text #8

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7426. Min Act: -0.1700

Data Index: 8352133 (Open Web Text)

Max Activating Token Index: 393

Text #9

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7339. Min Act: -0.1700

Data Index: 1938699 (Open Web Text)

Max Activating Token Index: 430

Text #10

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7311. Min Act: -0.1700

Data Index: 7399520 (Open Web Text)

Max Activating Token Index: 571

Text #11

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7137. Min Act: -0.1700

Data Index: 3458416 (Open Web Text)

Max Activating Token Index: 53

Text #12

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7211. Min Act: -0.1700

Data Index: 3215848 (Open Web Text)

Max Activating Token Index: 351

Text #13

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7187. Min Act: -0.1700

Data Index: 8395422 (Open Web Text)

Max Activating Token Index: 177

Text #14

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7158. Min Act: -0.1700

Data Index: 2809871 (Open Web Text)

Max Activating Token Index: 196

Text #15

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7072. Min Act: -0.1700

Data Index: 8228127 (Open Web Text)

Max Activating Token Index: 226

Text #16

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7080. Min Act: -0.1699

Data Index: 5944019 (Open Web Text)

Max Activating Token Index: 376

Text #17

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7081. Min Act: -0.1700

Data Index: 7963631 (Open Web Text)

Max Activating Token Index: 34

Text #18

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7092. Min Act: -0.1700

Data Index: 5124882 (Open Web Text)

Max Activating Token Index: 519

Text #19

Max Range: 1.7849. Min Range: -1.7849

Max Act: 1.7122. Min Act: -0.1700

Data Index: 3547434 (Open Web Text)

Max Activating Token Index: 979