Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1403 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3681. Min Act: -0.1700

Data Index: 4776984 (Open Web Text)

Max Activating Token Index: 850

Text #1

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3409. Min Act: -0.1700

Data Index: 2418364 (Open Web Text)

Max Activating Token Index: 389

Text #2

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3244. Min Act: -0.1700

Data Index: 6934752 (Open Web Text)

Max Activating Token Index: 650

Text #3

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3096. Min Act: -0.1700

Data Index: 5400916 (Open Web Text)

Max Activating Token Index: 684

Text #4

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3028. Min Act: -0.1700

Data Index: 6619360 (Open Web Text)

Max Activating Token Index: 809

Text #5

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3012. Min Act: -0.1700

Data Index: 2688289 (Open Web Text)

Max Activating Token Index: 542

Text #6

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2857. Min Act: -0.1700

Data Index: 8006605 (Open Web Text)

Max Activating Token Index: 919

Text #7

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3129. Min Act: -0.1700

Data Index: 7210348 (Open Web Text)

Max Activating Token Index: 772

Text #8

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3000. Min Act: -0.1700

Data Index: 3843448 (Open Web Text)

Max Activating Token Index: 1011

Text #9

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2975. Min Act: -0.1700

Data Index: 7001976 (Open Web Text)

Max Activating Token Index: 860

Text #10

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2751. Min Act: -0.1700

Data Index: 1457859 (Open Web Text)

Max Activating Token Index: 246

Text #11

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2863. Min Act: -0.1700

Data Index: 1557948 (Open Web Text)

Max Activating Token Index: 785

Text #12

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2522. Min Act: -0.1700

Data Index: 180334 (Open Web Text)

Max Activating Token Index: 524

Text #13

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2653. Min Act: -0.1700

Data Index: 8733445 (Open Web Text)

Max Activating Token Index: 260

Text #14

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2433. Min Act: -0.1700

Data Index: 7669979 (Open Web Text)

Max Activating Token Index: 614

Text #15

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2395. Min Act: -0.1700

Data Index: 2753848 (Open Web Text)

Max Activating Token Index: 102

Text #16

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2697. Min Act: -0.1700

Data Index: 4950076 (Open Web Text)

Max Activating Token Index: 792

Text #17

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2555. Min Act: -0.1700

Data Index: 4136227 (Open Web Text)

Max Activating Token Index: 531

Text #18

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2691. Min Act: -0.1700

Data Index: 4746566 (Open Web Text)

Max Activating Token Index: 402

Text #19

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2521. Min Act: -0.1700

Data Index: 5818057 (Open Web Text)

Max Activating Token Index: 33