Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 321 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.7968. Min Act: -0.1700

Data Index: 265964 (Open Web Text)

Max Activating Token Index: 90

Text #1

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.7505. Min Act: -0.1700

Data Index: 4597648 (Open Web Text)

Max Activating Token Index: 105

Text #2

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.7293. Min Act: -0.1700

Data Index: 5968586 (Open Web Text)

Max Activating Token Index: 91

Text #3

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.6711. Min Act: -0.1700

Data Index: 3519582 (Open Web Text)

Max Activating Token Index: 485

Text #4

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.6453. Min Act: -0.1700

Data Index: 2190045 (Open Web Text)

Max Activating Token Index: 399

Text #5

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.6197. Min Act: -0.1700

Data Index: 6682745 (Open Web Text)

Max Activating Token Index: 245

Text #6

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.6003. Min Act: -0.1700

Data Index: 6645037 (Open Web Text)

Max Activating Token Index: 294

Text #7

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5700. Min Act: -0.1700

Data Index: 2643287 (Open Web Text)

Max Activating Token Index: 228

Text #8

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5548. Min Act: -0.1700

Data Index: 5696519 (Open Web Text)

Max Activating Token Index: 879

Text #9

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5630. Min Act: -0.1700

Data Index: 7414746 (Open Web Text)

Max Activating Token Index: 273

Text #10

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5656. Min Act: -0.1700

Data Index: 5177562 (Open Web Text)

Max Activating Token Index: 239

Text #11

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5653. Min Act: -0.1700

Data Index: 1975581 (Open Web Text)

Max Activating Token Index: 121

Text #12

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5484. Min Act: -0.1700

Data Index: 4350473 (Open Web Text)

Max Activating Token Index: 273

Text #13

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5324. Min Act: -0.1700

Data Index: 2901329 (Open Web Text)

Max Activating Token Index: 116

Text #14

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5459. Min Act: -0.1700

Data Index: 1968928 (Open Web Text)

Max Activating Token Index: 628

Text #15

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5292. Min Act: -0.1700

Data Index: 5462819 (Open Web Text)

Max Activating Token Index: 905

Text #16

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5235. Min Act: -0.1700

Data Index: 561962 (Open Web Text)

Max Activating Token Index: 807

Text #17

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5243. Min Act: -0.1700

Data Index: 3817437 (Open Web Text)

Max Activating Token Index: 143

Text #18

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5220. Min Act: -0.1700

Data Index: 7965648 (Open Web Text)

Max Activating Token Index: 984

Text #19

Max Range: 3.7968. Min Range: -3.7968

Max Act: 3.5072. Min Act: -0.1700

Data Index: 6877423 (Open Web Text)

Max Activating Token Index: 804