Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2988 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.3783. Min Act: -0.1700

Data Index: 3756341 (Open Web Text)

Max Activating Token Index: 702

Text #1

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.3364. Min Act: -0.1700

Data Index: 2772702 (Open Web Text)

Max Activating Token Index: 263

Text #2

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2788. Min Act: -0.1700

Data Index: 1578561 (Open Web Text)

Max Activating Token Index: 248

Text #3

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2445. Min Act: -0.1700

Data Index: 6117004 (Open Web Text)

Max Activating Token Index: 846

Text #4

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2437. Min Act: -0.1700

Data Index: 1856103 (Open Web Text)

Max Activating Token Index: 815

Text #5

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2338. Min Act: -0.1700

Data Index: 4114573 (Open Web Text)

Max Activating Token Index: 592

Text #6

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2376. Min Act: -0.1700

Data Index: 6541374 (Open Web Text)

Max Activating Token Index: 909

Text #7

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2115. Min Act: -0.1700

Data Index: 256130 (Open Web Text)

Max Activating Token Index: 96

Text #8

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1914. Min Act: -0.1700

Data Index: 1903493 (Open Web Text)

Max Activating Token Index: 345

Text #9

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2064. Min Act: -0.1700

Data Index: 6103659 (Open Web Text)

Max Activating Token Index: 143

Text #10

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2044. Min Act: -0.1700

Data Index: 267507 (Open Web Text)

Max Activating Token Index: 283

Text #11

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1378. Min Act: -0.1700

Data Index: 3702365 (Open Web Text)

Max Activating Token Index: 371

Text #12

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1677. Min Act: -0.1700

Data Index: 2879629 (Open Web Text)

Max Activating Token Index: 321

Text #13

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.2249. Min Act: -0.1700

Data Index: 816164 (Open Web Text)

Max Activating Token Index: 738

Text #14

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1585. Min Act: -0.1700

Data Index: 8220672 (Open Web Text)

Max Activating Token Index: 629

Text #15

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1522. Min Act: -0.1700

Data Index: 1241153 (Open Web Text)

Max Activating Token Index: 381

Text #16

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1816. Min Act: -0.1700

Data Index: 6896064 (Open Web Text)

Max Activating Token Index: 835

Text #17

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1548. Min Act: -0.1700

Data Index: 3005361 (Open Web Text)

Max Activating Token Index: 407

Text #18

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1208. Min Act: -0.1700

Data Index: 5937450 (Open Web Text)

Max Activating Token Index: 364

Text #19

Max Range: 3.3783. Min Range: -3.3783

Max Act: 3.1335. Min Act: -0.1700

Data Index: 2960935 (Open Web Text)

Max Activating Token Index: 485