Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 515 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.4791. Min Act: -0.1700

Data Index: 4091280 (Open Web Text)

Max Activating Token Index: 433

Text #1

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.4137. Min Act: -0.1700

Data Index: 3389952 (Open Web Text)

Max Activating Token Index: 282

Text #2

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.3554. Min Act: -0.1700

Data Index: 4312793 (Open Web Text)

Max Activating Token Index: 155

Text #3

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.3476. Min Act: -0.1700

Data Index: 1148451 (Open Web Text)

Max Activating Token Index: 191

Text #4

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.3570. Min Act: -0.1700

Data Index: 4559944 (Open Web Text)

Max Activating Token Index: 523

Text #5

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.3108. Min Act: -0.1700

Data Index: 3440116 (Open Web Text)

Max Activating Token Index: 356

Text #6

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.3201. Min Act: -0.1700

Data Index: 6081423 (Open Web Text)

Max Activating Token Index: 310

Text #7

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2883. Min Act: -0.1700

Data Index: 1296278 (Open Web Text)

Max Activating Token Index: 375

Text #8

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2866. Min Act: -0.1700

Data Index: 5782826 (Open Web Text)

Max Activating Token Index: 346

Text #9

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2801. Min Act: -0.1700

Data Index: 6438115 (Open Web Text)

Max Activating Token Index: 418

Text #10

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2877. Min Act: -0.1700

Data Index: 6630403 (Open Web Text)

Max Activating Token Index: 302

Text #11

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2691. Min Act: -0.1700

Data Index: 8062388 (Open Web Text)

Max Activating Token Index: 661

Text #12

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2611. Min Act: -0.1700

Data Index: 5782823 (Open Web Text)

Max Activating Token Index: 702

Text #13

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2748. Min Act: -0.1700

Data Index: 2238929 (Open Web Text)

Max Activating Token Index: 301

Text #14

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2549. Min Act: -0.1700

Data Index: 2784840 (Open Web Text)

Max Activating Token Index: 221

Text #15

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2512. Min Act: -0.1700

Data Index: 6133252 (Open Web Text)

Max Activating Token Index: 531

Text #16

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2497. Min Act: -0.1700

Data Index: 1981320 (Open Web Text)

Max Activating Token Index: 566

Text #17

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2487. Min Act: -0.1700

Data Index: 1296277 (Open Web Text)

Max Activating Token Index: 522

Text #18

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2344. Min Act: -0.1700

Data Index: 6161924 (Open Web Text)

Max Activating Token Index: 287

Text #19

Max Range: 3.4791. Min Range: -3.4791

Max Act: 3.2477. Min Act: -0.1700

Data Index: 6076341 (Open Web Text)

Max Activating Token Index: 442