Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1859 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0315. Min Range: -2.0315

Max Act: 2.0315. Min Act: -0.1672

Data Index: 6496580 (Open Web Text)

Max Activating Token Index: 532

Text #1

Max Range: 2.0315. Min Range: -2.0315

Max Act: 2.0078. Min Act: -0.1658

Data Index: 2073455 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9570. Min Act: -0.1652

Data Index: 6728477 (Open Web Text)

Max Activating Token Index: 281

Text #3

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9664. Min Act: -0.1615

Data Index: 3928750 (Open Web Text)

Max Activating Token Index: 305

Text #4

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9506. Min Act: -0.1613

Data Index: 2000801 (Open Web Text)

Max Activating Token Index: 610

Text #5

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9512. Min Act: -0.1544

Data Index: 8260077 (Open Web Text)

Max Activating Token Index: 1013

Text #6

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9591. Min Act: -0.1539

Data Index: 723869 (Open Web Text)

Max Activating Token Index: 870

Text #7

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9442. Min Act: -0.1628

Data Index: 5708978 (Open Web Text)

Max Activating Token Index: 332

Text #8

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9366. Min Act: -0.1672

Data Index: 4130984 (Open Web Text)

Max Activating Token Index: 299

Text #9

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9479. Min Act: -0.1589

Data Index: 5533693 (Open Web Text)

Max Activating Token Index: 356

Text #10

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9416. Min Act: -0.1588

Data Index: 4809594 (Open Web Text)

Max Activating Token Index: 540

Text #11

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9245. Min Act: -0.1586

Data Index: 837903 (Open Web Text)

Max Activating Token Index: 290

Text #12

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9321. Min Act: -0.1595

Data Index: 5017738 (Open Web Text)

Max Activating Token Index: 216

Text #13

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9323. Min Act: -0.1353

Data Index: 4695260 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9265. Min Act: -0.1668

Data Index: 4844264 (Open Web Text)

Max Activating Token Index: 289

Text #15

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9174. Min Act: -0.1435

Data Index: 2979175 (Open Web Text)

Max Activating Token Index: 256

Text #16

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9308. Min Act: -0.0986

Data Index: 4688292 (Open Web Text)

Max Activating Token Index: 563

Text #17

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9193. Min Act: -0.1614

Data Index: 2302607 (Open Web Text)

Max Activating Token Index: 297

Text #18

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9190. Min Act: -0.1682

Data Index: 1508471 (Open Web Text)

Max Activating Token Index: 359

Text #19

Max Range: 2.0315. Min Range: -2.0315

Max Act: 1.9277. Min Act: -0.1610

Data Index: 2326433 (Open Web Text)

Max Activating Token Index: 1002