Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2578 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.4936. Min Act: -0.1700

Data Index: 359650 (Open Web Text)

Max Activating Token Index: 655

Text #1

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.4857. Min Act: -0.1700

Data Index: 5045413 (Open Web Text)

Max Activating Token Index: 184

Text #2

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.2944. Min Act: -0.1700

Data Index: 6137004 (Open Web Text)

Max Activating Token Index: 230

Text #3

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.2639. Min Act: -0.1700

Data Index: 206270 (Open Web Text)

Max Activating Token Index: 263

Text #4

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.2444. Min Act: -0.1700

Data Index: 6038655 (Open Web Text)

Max Activating Token Index: 206

Text #5

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.2465. Min Act: -0.1700

Data Index: 5805080 (Open Web Text)

Max Activating Token Index: 28

Text #6

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.2182. Min Act: -0.1700

Data Index: 6634976 (Open Web Text)

Max Activating Token Index: 612

Text #7

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1506. Min Act: -0.1700

Data Index: 6210026 (Open Web Text)

Max Activating Token Index: 788

Text #8

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1371. Min Act: -0.1700

Data Index: 2028407 (Open Web Text)

Max Activating Token Index: 406

Text #9

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1474. Min Act: -0.1700

Data Index: 4092803 (Open Web Text)

Max Activating Token Index: 667

Text #10

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1336. Min Act: -0.1700

Data Index: 4761636 (Open Web Text)

Max Activating Token Index: 899

Text #11

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1375. Min Act: -0.1700

Data Index: 3811475 (Open Web Text)

Max Activating Token Index: 240

Text #12

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1350. Min Act: -0.1700

Data Index: 8171416 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1040. Min Act: -0.1700

Data Index: 5863926 (Open Web Text)

Max Activating Token Index: 1004

Text #14

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.0657. Min Act: -0.1700

Data Index: 1981909 (Open Web Text)

Max Activating Token Index: 365

Text #15

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.0899. Min Act: -0.1700

Data Index: 4141480 (Open Web Text)

Max Activating Token Index: 262

Text #16

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.1033. Min Act: -0.1700

Data Index: 5383900 (Open Web Text)

Max Activating Token Index: 614

Text #17

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.0707. Min Act: -0.1700

Data Index: 6061850 (Open Web Text)

Max Activating Token Index: 243

Text #18

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.0941. Min Act: -0.1700

Data Index: 5543733 (Open Web Text)

Max Activating Token Index: 526

Text #19

Max Range: 4.4936. Min Range: -4.4936

Max Act: 4.0761. Min Act: -0.1700

Data Index: 7653284 (Open Web Text)

Max Activating Token Index: 653