Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1478 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.5864. Min Act: -0.1700

Data Index: 4889677 (Open Web Text)

Max Activating Token Index: 249

Text #1

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.5852. Min Act: -0.1700

Data Index: 1954632 (Open Web Text)

Max Activating Token Index: 633

Text #2

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.5933. Min Act: -0.1700

Data Index: 6039807 (Open Web Text)

Max Activating Token Index: 732

Text #3

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.4621. Min Act: -0.1700

Data Index: 4856794 (Open Web Text)

Max Activating Token Index: 536

Text #4

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.4581. Min Act: -0.1700

Data Index: 5376181 (Open Web Text)

Max Activating Token Index: 693

Text #5

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.4062. Min Act: -0.1700

Data Index: 6607596 (Open Web Text)

Max Activating Token Index: 843

Text #6

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3766. Min Act: -0.1700

Data Index: 8104530 (Open Web Text)

Max Activating Token Index: 1017

Text #7

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3568. Min Act: -0.1700

Data Index: 6281655 (Open Web Text)

Max Activating Token Index: 293

Text #8

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3710. Min Act: -0.1700

Data Index: 2270826 (Open Web Text)

Max Activating Token Index: 349

Text #9

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3838. Min Act: -0.1700

Data Index: 5295158 (Open Web Text)

Max Activating Token Index: 36

Text #10

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3523. Min Act: -0.1700

Data Index: 5005250 (Open Web Text)

Max Activating Token Index: 654

Text #11

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3300. Min Act: -0.1700

Data Index: 412354 (Open Web Text)

Max Activating Token Index: 681

Text #12

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3227. Min Act: -0.1700

Data Index: 4345946 (Open Web Text)

Max Activating Token Index: 716

Text #13

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2494. Min Act: -0.1700

Data Index: 637251 (Open Web Text)

Max Activating Token Index: 1015

Text #14

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.3137. Min Act: -0.1700

Data Index: 7466976 (Open Web Text)

Max Activating Token Index: 352

Text #15

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2836. Min Act: -0.1700

Data Index: 4981310 (Open Web Text)

Max Activating Token Index: 231

Text #16

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2457. Min Act: -0.1700

Data Index: 4718208 (Open Web Text)

Max Activating Token Index: 162

Text #17

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2567. Min Act: -0.1700

Data Index: 308834 (Open Web Text)

Max Activating Token Index: 813

Text #18

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2490. Min Act: -0.1700

Data Index: 6607610 (Open Web Text)

Max Activating Token Index: 111

Text #19

Max Range: 5.5933. Min Range: -5.5933

Max Act: 5.2323. Min Act: -0.1700

Data Index: 4883503 (Open Web Text)

Max Activating Token Index: 517