Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 710 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.3687. Min Act: -0.1700

Data Index: 7680855 (Open Web Text)

Max Activating Token Index: 474

Text #1

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.3068. Min Act: -0.1700

Data Index: 7680831 (Open Web Text)

Max Activating Token Index: 196

Text #2

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.2488. Min Act: -0.1700

Data Index: 6575489 (Open Web Text)

Max Activating Token Index: 363

Text #3

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.2299. Min Act: -0.1700

Data Index: 1639522 (Open Web Text)

Max Activating Token Index: 624

Text #4

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.2140. Min Act: -0.1700

Data Index: 7106603 (Open Web Text)

Max Activating Token Index: 481

Text #5

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1918. Min Act: -0.1700

Data Index: 5696347 (Open Web Text)

Max Activating Token Index: 747

Text #6

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.2024. Min Act: -0.1700

Data Index: 5989323 (Open Web Text)

Max Activating Token Index: 360

Text #7

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1812. Min Act: -0.1700

Data Index: 7914537 (Open Web Text)

Max Activating Token Index: 199

Text #8

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1617. Min Act: -0.1700

Data Index: 5630853 (Open Web Text)

Max Activating Token Index: 337

Text #9

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1658. Min Act: -0.1700

Data Index: 6263456 (Open Web Text)

Max Activating Token Index: 217

Text #10

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1647. Min Act: -0.1700

Data Index: 288440 (Open Web Text)

Max Activating Token Index: 175

Text #11

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1700. Min Act: -0.1700

Data Index: 26026 (Open Web Text)

Max Activating Token Index: 653

Text #12

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1546. Min Act: -0.1700

Data Index: 1135561 (Open Web Text)

Max Activating Token Index: 185

Text #13

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1547. Min Act: -0.1700

Data Index: 4398200 (Open Web Text)

Max Activating Token Index: 733

Text #14

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1382. Min Act: -0.1700

Data Index: 250129 (Open Web Text)

Max Activating Token Index: 303

Text #15

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1343. Min Act: -0.1700

Data Index: 2588239 (Open Web Text)

Max Activating Token Index: 963

Text #16

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1301. Min Act: -0.1700

Data Index: 6043632 (Open Web Text)

Max Activating Token Index: 116

Text #17

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1399. Min Act: -0.1700

Data Index: 6043619 (Open Web Text)

Max Activating Token Index: 598

Text #18

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1458. Min Act: -0.1700

Data Index: 2086117 (Open Web Text)

Max Activating Token Index: 502

Text #19

Max Range: 3.3687. Min Range: -3.3687

Max Act: 3.1047. Min Act: -0.1700

Data Index: 4718058 (Open Web Text)

Max Activating Token Index: 1010