Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1407 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.8493. Min Act: -0.1700

Data Index: 3150203 (Open Web Text)

Max Activating Token Index: 50

Text #1

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.5994. Min Act: -0.1700

Data Index: 2751265 (Open Web Text)

Max Activating Token Index: 55

Text #2

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.4798. Min Act: -0.1700

Data Index: 4600748 (Open Web Text)

Max Activating Token Index: 45

Text #3

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.3434. Min Act: -0.1700

Data Index: 1142625 (Open Web Text)

Max Activating Token Index: 167

Text #4

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.3029. Min Act: -0.1700

Data Index: 4942675 (Open Web Text)

Max Activating Token Index: 737

Text #5

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.2654. Min Act: -0.1700

Data Index: 1683475 (Open Web Text)

Max Activating Token Index: 398

Text #6

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.2266. Min Act: -0.1700

Data Index: 8477887 (Open Web Text)

Max Activating Token Index: 151

Text #7

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.1900. Min Act: -0.1700

Data Index: 3254968 (Open Web Text)

Max Activating Token Index: 92

Text #8

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.2125. Min Act: -0.1700

Data Index: 8670503 (Open Web Text)

Max Activating Token Index: 33

Text #9

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.1608. Min Act: -0.1700

Data Index: 6044919 (Open Web Text)

Max Activating Token Index: 411

Text #10

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0991. Min Act: -0.1700

Data Index: 1074692 (Open Web Text)

Max Activating Token Index: 967

Text #11

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0992. Min Act: -0.1700

Data Index: 1452236 (Open Web Text)

Max Activating Token Index: 95

Text #12

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.1167. Min Act: -0.1700

Data Index: 8289311 (Open Web Text)

Max Activating Token Index: 540

Text #13

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0529. Min Act: -0.1700

Data Index: 1331603 (Open Web Text)

Max Activating Token Index: 309

Text #14

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0471. Min Act: -0.1700

Data Index: 3624526 (Open Web Text)

Max Activating Token Index: 153

Text #15

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0417. Min Act: -0.1700

Data Index: 2693973 (Open Web Text)

Max Activating Token Index: 48

Text #16

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0270. Min Act: -0.1700

Data Index: 2010196 (Open Web Text)

Max Activating Token Index: 204

Text #17

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0396. Min Act: -0.1700

Data Index: 3700685 (Open Web Text)

Max Activating Token Index: 80

Text #18

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0028. Min Act: -0.1700

Data Index: 3823334 (Open Web Text)

Max Activating Token Index: 94

Text #19

Max Range: 6.8493. Min Range: -6.8493

Max Act: 6.0209. Min Act: -0.1700

Data Index: 5795441 (Open Web Text)

Max Activating Token Index: 38