Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 861 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1870. Min Act: -0.1700

Data Index: 2420922 (Open Web Text)

Max Activating Token Index: 947

Text #1

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1517. Min Act: -0.1700

Data Index: 4656027 (Open Web Text)

Max Activating Token Index: 509

Text #2

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1196. Min Act: -0.1700

Data Index: 4855319 (Open Web Text)

Max Activating Token Index: 132

Text #3

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1031. Min Act: -0.1700

Data Index: 5712838 (Open Web Text)

Max Activating Token Index: 57

Text #4

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1269. Min Act: -0.1700

Data Index: 4656026 (Open Web Text)

Max Activating Token Index: 334

Text #5

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.1399. Min Act: -0.1700

Data Index: 5712837 (Open Web Text)

Max Activating Token Index: 626

Text #6

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.0575. Min Act: -0.1700

Data Index: 7028081 (Open Web Text)

Max Activating Token Index: 543

Text #7

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.0557. Min Act: -0.1700

Data Index: 5173351 (Open Web Text)

Max Activating Token Index: 777

Text #8

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9979. Min Act: -0.1700

Data Index: 5789381 (Open Web Text)

Max Activating Token Index: 163

Text #9

Max Range: 4.1870. Min Range: -4.1870

Max Act: 4.0291. Min Act: -0.1700

Data Index: 2423473 (Open Web Text)

Max Activating Token Index: 942

Text #10

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.8551. Min Act: -0.1700

Data Index: 6964706 (Open Web Text)

Max Activating Token Index: 633

Text #11

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9736. Min Act: -0.1700

Data Index: 5105812 (Open Web Text)

Max Activating Token Index: 316

Text #12

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9706. Min Act: -0.1700

Data Index: 5736108 (Open Web Text)

Max Activating Token Index: 47

Text #13

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9417. Min Act: -0.1700

Data Index: 4656025 (Open Web Text)

Max Activating Token Index: 182

Text #14

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9499. Min Act: -0.1700

Data Index: 6514833 (Open Web Text)

Max Activating Token Index: 556

Text #15

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9413. Min Act: -0.1700

Data Index: 6702865 (Open Web Text)

Max Activating Token Index: 186

Text #16

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9571. Min Act: -0.1700

Data Index: 3534481 (Open Web Text)

Max Activating Token Index: 533

Text #17

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9385. Min Act: -0.1700

Data Index: 7584191 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9032. Min Act: -0.1700

Data Index: 1720506 (Open Web Text)

Max Activating Token Index: 77

Text #19

Max Range: 4.1870. Min Range: -4.1870

Max Act: 3.9251. Min Act: -0.1700

Data Index: 22412 (Open Web Text)

Max Activating Token Index: 782