Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2633 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.3459. Min Act: -0.1450

Data Index: 4717017 (Open Web Text)

Max Activating Token Index: 724

Text #1

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.2598. Min Act: -0.0594

Data Index: 3879769 (Open Web Text)

Max Activating Token Index: 510

Text #2

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.2418. Min Act: -0.1692

Data Index: 5239239 (Open Web Text)

Max Activating Token Index: 551

Text #3

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1824. Min Act: -0.1685

Data Index: 7177532 (Open Web Text)

Max Activating Token Index: 43

Text #4

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1515. Min Act: -0.1373

Data Index: 3027008 (Open Web Text)

Max Activating Token Index: 774

Text #5

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1483. Min Act: -0.1651

Data Index: 3879770 (Open Web Text)

Max Activating Token Index: 112

Text #6

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1461. Min Act: -0.1700

Data Index: 4842948 (Open Web Text)

Max Activating Token Index: 31

Text #7

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1382. Min Act: -0.0944

Data Index: 529132 (Open Web Text)

Max Activating Token Index: 890

Text #8

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1150. Min Act: -0.1513

Data Index: 4979158 (Open Web Text)

Max Activating Token Index: 678

Text #9

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1248. Min Act: -0.1257

Data Index: 6986802 (Open Web Text)

Max Activating Token Index: 905

Text #10

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1145. Min Act: -0.1377

Data Index: 1384254 (Open Web Text)

Max Activating Token Index: 135

Text #11

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1135. Min Act: -0.1691

Data Index: 8429012 (Open Web Text)

Max Activating Token Index: 23

Text #12

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.1158. Min Act: -0.1621

Data Index: 3945318 (Open Web Text)

Max Activating Token Index: 608

Text #13

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0955. Min Act: -0.1625

Data Index: 2201233 (Open Web Text)

Max Activating Token Index: 125

Text #14

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0735. Min Act: -0.0449

Data Index: 5143754 (Open Web Text)

Max Activating Token Index: 585

Text #15

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0796. Min Act: -0.1606

Data Index: 5565135 (Open Web Text)

Max Activating Token Index: 297

Text #16

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0728. Min Act: -0.1565

Data Index: 6846257 (Open Web Text)

Max Activating Token Index: 347

Text #17

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0767. Min Act: -0.0926

Data Index: 7600905 (Open Web Text)

Max Activating Token Index: 362

Text #18

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0846. Min Act: -0.1652

Data Index: 8427367 (Open Web Text)

Max Activating Token Index: 381

Text #19

Max Range: 1.3459. Min Range: -1.3459

Max Act: 1.0721. Min Act: -0.1245

Data Index: 7177531 (Open Web Text)

Max Activating Token Index: 850