Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2925 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.4472. Min Act: -0.1700

Data Index: 7931397 (Open Web Text)

Max Activating Token Index: 366

Text #1

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.1711. Min Act: -0.1700

Data Index: 6612611 (Open Web Text)

Max Activating Token Index: 922

Text #2

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.2113. Min Act: -0.1700

Data Index: 5739252 (Open Web Text)

Max Activating Token Index: 160

Text #3

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.1419. Min Act: -0.1700

Data Index: 5479645 (Open Web Text)

Max Activating Token Index: 626

Text #4

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.1135. Min Act: -0.1700

Data Index: 3516413 (Open Web Text)

Max Activating Token Index: 316

Text #5

Max Range: 4.4472. Min Range: -4.4472

Max Act: 4.0251. Min Act: -0.1700

Data Index: 7191153 (Open Web Text)

Max Activating Token Index: 140

Text #6

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.9959. Min Act: -0.1700

Data Index: 4905262 (Open Web Text)

Max Activating Token Index: 569

Text #7

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.9215. Min Act: -0.1700

Data Index: 5654323 (Open Web Text)

Max Activating Token Index: 546

Text #8

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.8759. Min Act: -0.1700

Data Index: 3781180 (Open Web Text)

Max Activating Token Index: 101

Text #9

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.9189. Min Act: -0.1700

Data Index: 251436 (Open Web Text)

Max Activating Token Index: 919

Text #10

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.8994. Min Act: -0.1700

Data Index: 2154356 (Open Web Text)

Max Activating Token Index: 708

Text #11

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.8124. Min Act: -0.1700

Data Index: 6074210 (Open Web Text)

Max Activating Token Index: 863

Text #12

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.9322. Min Act: -0.1700

Data Index: 2010787 (Open Web Text)

Max Activating Token Index: 960

Text #13

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.9170. Min Act: -0.1700

Data Index: 1745693 (Open Web Text)

Max Activating Token Index: 667

Text #14

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.7293. Min Act: -0.1700

Data Index: 18958 (Open Web Text)

Max Activating Token Index: 142

Text #15

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.7197. Min Act: -0.1700

Data Index: 5391196 (Open Web Text)

Max Activating Token Index: 579

Text #16

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.7722. Min Act: -0.1700

Data Index: 2370515 (Open Web Text)

Max Activating Token Index: 417

Text #17

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.7299. Min Act: -0.1700

Data Index: 2506724 (Open Web Text)

Max Activating Token Index: 325

Text #18

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.7288. Min Act: -0.1700

Data Index: 1025047 (Open Web Text)

Max Activating Token Index: 982

Text #19

Max Range: 4.4472. Min Range: -4.4472

Max Act: 3.6896. Min Act: -0.1700

Data Index: 6369108 (Open Web Text)

Max Activating Token Index: 849