Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1165 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.7551. Min Act: -0.1700

Data Index: 257046 (Open Web Text)

Max Activating Token Index: 118

Text #1

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.7209. Min Act: -0.1700

Data Index: 8799251 (Open Web Text)

Max Activating Token Index: 74

Text #2

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.7027. Min Act: -0.1700

Data Index: 3800212 (Open Web Text)

Max Activating Token Index: 186

Text #3

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.7178. Min Act: -0.1700

Data Index: 7066810 (Open Web Text)

Max Activating Token Index: 725

Text #4

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.7139. Min Act: -0.1700

Data Index: 2063017 (Open Web Text)

Max Activating Token Index: 243

Text #5

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6726. Min Act: -0.1700

Data Index: 2540915 (Open Web Text)

Max Activating Token Index: 75

Text #6

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6781. Min Act: -0.1700

Data Index: 927046 (Open Web Text)

Max Activating Token Index: 243

Text #7

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6697. Min Act: -0.1700

Data Index: 226949 (Open Web Text)

Max Activating Token Index: 236

Text #8

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6950. Min Act: -0.1700

Data Index: 7428363 (Open Web Text)

Max Activating Token Index: 624

Text #9

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6511. Min Act: -0.1700

Data Index: 5479366 (Open Web Text)

Max Activating Token Index: 154

Text #10

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6565. Min Act: -0.1700

Data Index: 114695 (Open Web Text)

Max Activating Token Index: 238

Text #11

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6544. Min Act: -0.1695

Data Index: 2152986 (Open Web Text)

Max Activating Token Index: 384

Text #12

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6466. Min Act: -0.1700

Data Index: 1967514 (Open Web Text)

Max Activating Token Index: 66

Text #13

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6321. Min Act: -0.1700

Data Index: 744576 (Open Web Text)

Max Activating Token Index: 200

Text #14

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6597. Min Act: -0.1700

Data Index: 7008197 (Open Web Text)

Max Activating Token Index: 144

Text #15

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6530. Min Act: -0.1700

Data Index: 5957725 (Open Web Text)

Max Activating Token Index: 267

Text #16

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6479. Min Act: -0.1700

Data Index: 5754106 (Open Web Text)

Max Activating Token Index: 700

Text #17

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6109. Min Act: -0.1700

Data Index: 1562354 (Open Web Text)

Max Activating Token Index: 426

Text #18

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6253. Min Act: -0.1700

Data Index: 4012904 (Open Web Text)

Max Activating Token Index: 498

Text #19

Max Range: 3.7551. Min Range: -3.7551

Max Act: 3.6085. Min Act: -0.1700

Data Index: 894267 (Open Web Text)

Max Activating Token Index: 272