Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1435 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.3311. Min Act: -0.1700

Data Index: 1441616 (Open Web Text)

Max Activating Token Index: 157

Text #1

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.2728. Min Act: -0.1700

Data Index: 8765631 (Open Web Text)

Max Activating Token Index: 127

Text #2

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.2481. Min Act: -0.1700

Data Index: 7410199 (Open Web Text)

Max Activating Token Index: 320

Text #3

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.2338. Min Act: -0.1700

Data Index: 1813296 (Open Web Text)

Max Activating Token Index: 532

Text #4

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.2009. Min Act: -0.1700

Data Index: 5825542 (Open Web Text)

Max Activating Token Index: 562

Text #5

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1914. Min Act: -0.1700

Data Index: 7788704 (Open Web Text)

Max Activating Token Index: 388

Text #6

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1740. Min Act: -0.1700

Data Index: 3877549 (Open Web Text)

Max Activating Token Index: 214

Text #7

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1504. Min Act: -0.1700

Data Index: 4248593 (Open Web Text)

Max Activating Token Index: 326

Text #8

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1420. Min Act: -0.1700

Data Index: 4425547 (Open Web Text)

Max Activating Token Index: 358

Text #9

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1149. Min Act: -0.1700

Data Index: 5768671 (Open Web Text)

Max Activating Token Index: 147

Text #10

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1228. Min Act: -0.1700

Data Index: 1183327 (Open Web Text)

Max Activating Token Index: 931

Text #11

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.1088. Min Act: -0.1700

Data Index: 5580214 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0813. Min Act: -0.1700

Data Index: 3908838 (Open Web Text)

Max Activating Token Index: 345

Text #13

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0830. Min Act: -0.1700

Data Index: 1731094 (Open Web Text)

Max Activating Token Index: 69

Text #14

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0741. Min Act: -0.1700

Data Index: 1813298 (Open Web Text)

Max Activating Token Index: 444

Text #15

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0456. Min Act: -0.1700

Data Index: 4829405 (Open Web Text)

Max Activating Token Index: 202

Text #16

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0573. Min Act: -0.1700

Data Index: 2489115 (Open Web Text)

Max Activating Token Index: 224

Text #17

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0571. Min Act: -0.1700

Data Index: 6273387 (Open Web Text)

Max Activating Token Index: 213

Text #18

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0371. Min Act: -0.1700

Data Index: 4739483 (Open Web Text)

Max Activating Token Index: 763

Text #19

Max Range: 3.3311. Min Range: -3.3311

Max Act: 3.0347. Min Act: -0.1700

Data Index: 3591425 (Open Web Text)

Max Activating Token Index: 293