Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2397 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.9054. Min Act: -0.1700

Data Index: 7687230 (Open Web Text)

Max Activating Token Index: 134

Text #1

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7881. Min Act: -0.1700

Data Index: 1607360 (Open Web Text)

Max Activating Token Index: 503

Text #2

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7605. Min Act: -0.1700

Data Index: 4336997 (Open Web Text)

Max Activating Token Index: 77

Text #3

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7391. Min Act: -0.1700

Data Index: 6541316 (Open Web Text)

Max Activating Token Index: 375

Text #4

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7281. Min Act: -0.1700

Data Index: 8695757 (Open Web Text)

Max Activating Token Index: 320

Text #5

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7185. Min Act: -0.1700

Data Index: 1185212 (Open Web Text)

Max Activating Token Index: 900

Text #6

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7214. Min Act: -0.1700

Data Index: 908096 (Open Web Text)

Max Activating Token Index: 36

Text #7

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.7397. Min Act: -0.1700

Data Index: 1418557 (Open Web Text)

Max Activating Token Index: 294

Text #8

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6945. Min Act: -0.1700

Data Index: 5012119 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6966. Min Act: -0.1700

Data Index: 7802757 (Open Web Text)

Max Activating Token Index: 478

Text #10

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6669. Min Act: -0.1700

Data Index: 6385008 (Open Web Text)

Max Activating Token Index: 634

Text #11

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6598. Min Act: -0.1700

Data Index: 5437565 (Open Web Text)

Max Activating Token Index: 133

Text #12

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6830. Min Act: -0.1700

Data Index: 1363251 (Open Web Text)

Max Activating Token Index: 284

Text #13

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6424. Min Act: -0.1700

Data Index: 4440724 (Open Web Text)

Max Activating Token Index: 94

Text #14

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6242. Min Act: -0.1700

Data Index: 4573004 (Open Web Text)

Max Activating Token Index: 145

Text #15

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6598. Min Act: -0.1700

Data Index: 4983955 (Open Web Text)

Max Activating Token Index: 469

Text #16

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6353. Min Act: -0.1700

Data Index: 1181666 (Open Web Text)

Max Activating Token Index: 241

Text #17

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6244. Min Act: -0.1700

Data Index: 6802932 (Open Web Text)

Max Activating Token Index: 202

Text #18

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6456. Min Act: -0.1700

Data Index: 5421435 (Open Web Text)

Max Activating Token Index: 48

Text #19

Max Range: 3.9054. Min Range: -3.9054

Max Act: 3.6338. Min Act: -0.1700

Data Index: 7479633 (Open Web Text)

Max Activating Token Index: 70