Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 211 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.5398. Min Act: -0.1700

Data Index: 7314189 (Open Web Text)

Max Activating Token Index: 42

Text #1

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.3484. Min Act: -0.1700

Data Index: 6692218 (Open Web Text)

Max Activating Token Index: 257

Text #2

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.2620. Min Act: -0.1700

Data Index: 3591870 (Open Web Text)

Max Activating Token Index: 471

Text #3

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.2595. Min Act: -0.1700

Data Index: 4031848 (Open Web Text)

Max Activating Token Index: 562

Text #4

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1885. Min Act: -0.1700

Data Index: 6573666 (Open Web Text)

Max Activating Token Index: 533

Text #5

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1736. Min Act: -0.1700

Data Index: 4165677 (Open Web Text)

Max Activating Token Index: 896

Text #6

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.2043. Min Act: -0.1700

Data Index: 3514029 (Open Web Text)

Max Activating Token Index: 936

Text #7

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1491. Min Act: -0.1700

Data Index: 2008809 (Open Web Text)

Max Activating Token Index: 1015

Text #8

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1230. Min Act: -0.1700

Data Index: 5344905 (Open Web Text)

Max Activating Token Index: 1008

Text #9

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.0928. Min Act: -0.1700

Data Index: 293249 (Open Web Text)

Max Activating Token Index: 249

Text #10

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1291. Min Act: -0.1700

Data Index: 5147589 (Open Web Text)

Max Activating Token Index: 430

Text #11

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.0404. Min Act: -0.1700

Data Index: 5675875 (Open Web Text)

Max Activating Token Index: 529

Text #12

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1486. Min Act: -0.1700

Data Index: 6109783 (Open Web Text)

Max Activating Token Index: 965

Text #13

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1112. Min Act: -0.1700

Data Index: 8342026 (Open Web Text)

Max Activating Token Index: 928

Text #14

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1499. Min Act: -0.1700

Data Index: 6278069 (Open Web Text)

Max Activating Token Index: 557

Text #15

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1101. Min Act: -0.1700

Data Index: 933642 (Open Web Text)

Max Activating Token Index: 845

Text #16

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.1477. Min Act: -0.1700

Data Index: 5502531 (Open Web Text)

Max Activating Token Index: 650

Text #17

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.0557. Min Act: -0.1700

Data Index: 8255451 (Open Web Text)

Max Activating Token Index: 396

Text #18

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.0979. Min Act: -0.1700

Data Index: 1114308 (Open Web Text)

Max Activating Token Index: 88

Text #19

Max Range: 3.5398. Min Range: -3.5398

Max Act: 3.0669. Min Act: -0.1700

Data Index: 2632684 (Open Web Text)

Max Activating Token Index: 707