Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1733 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.4947. Min Act: -0.1700

Data Index: 8261285 (Open Web Text)

Max Activating Token Index: 262

Text #1

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.4536. Min Act: -0.1700

Data Index: 8665241 (Open Web Text)

Max Activating Token Index: 128

Text #2

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.4440. Min Act: -0.1700

Data Index: 8665239 (Open Web Text)

Max Activating Token Index: 100

Text #3

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.4033. Min Act: -0.1700

Data Index: 7591746 (Open Web Text)

Max Activating Token Index: 890

Text #4

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.4284. Min Act: -0.1700

Data Index: 7420828 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3925. Min Act: -0.1700

Data Index: 32013 (Open Web Text)

Max Activating Token Index: 815

Text #6

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3544. Min Act: -0.1700

Data Index: 8382605 (Open Web Text)

Max Activating Token Index: 30

Text #7

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3770. Min Act: -0.1700

Data Index: 8617682 (Open Web Text)

Max Activating Token Index: 372

Text #8

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3805. Min Act: -0.1700

Data Index: 478814 (Open Web Text)

Max Activating Token Index: 827

Text #9

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3435. Min Act: -0.1700

Data Index: 8665240 (Open Web Text)

Max Activating Token Index: 196

Text #10

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3568. Min Act: -0.1700

Data Index: 6250843 (Open Web Text)

Max Activating Token Index: 130

Text #11

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3197. Min Act: -0.1700

Data Index: 8382604 (Open Web Text)

Max Activating Token Index: 861

Text #12

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3394. Min Act: -0.1700

Data Index: 7709709 (Open Web Text)

Max Activating Token Index: 205

Text #13

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3121. Min Act: -0.1700

Data Index: 5417578 (Open Web Text)

Max Activating Token Index: 932

Text #14

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3286. Min Act: -0.1700

Data Index: 3781649 (Open Web Text)

Max Activating Token Index: 121

Text #15

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.2777. Min Act: -0.1700

Data Index: 4222587 (Open Web Text)

Max Activating Token Index: 875

Text #16

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.2906. Min Act: -0.1700

Data Index: 5417580 (Open Web Text)

Max Activating Token Index: 363

Text #17

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.3098. Min Act: -0.1700

Data Index: 1220970 (Open Web Text)

Max Activating Token Index: 128

Text #18

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.2866. Min Act: -0.1700

Data Index: 4055875 (Open Web Text)

Max Activating Token Index: 663

Text #19

Max Range: 4.4947. Min Range: -4.4947

Max Act: 4.2464. Min Act: -0.1700

Data Index: 6987041 (Open Web Text)

Max Activating Token Index: 928