Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1144 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3659. Min Act: -0.1700

Data Index: 1670695 (Open Web Text)

Max Activating Token Index: 101

Text #1

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3522. Min Act: -0.1700

Data Index: 7859873 (Open Web Text)

Max Activating Token Index: 187

Text #2

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3190. Min Act: -0.1700

Data Index: 5898936 (Open Web Text)

Max Activating Token Index: 750

Text #3

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3263. Min Act: -0.1700

Data Index: 1555326 (Open Web Text)

Max Activating Token Index: 590

Text #4

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3081. Min Act: -0.1700

Data Index: 699171 (Open Web Text)

Max Activating Token Index: 434

Text #5

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3093. Min Act: -0.1700

Data Index: 2134832 (Open Web Text)

Max Activating Token Index: 567

Text #6

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.3165. Min Act: -0.1700

Data Index: 7115019 (Open Web Text)

Max Activating Token Index: 699

Text #7

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2834. Min Act: -0.1700

Data Index: 3068925 (Open Web Text)

Max Activating Token Index: 732

Text #8

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2916. Min Act: -0.1700

Data Index: 3564015 (Open Web Text)

Max Activating Token Index: 147

Text #9

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2948. Min Act: -0.1700

Data Index: 485391 (Open Web Text)

Max Activating Token Index: 832

Text #10

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2978. Min Act: -0.1700

Data Index: 6825988 (Open Web Text)

Max Activating Token Index: 220

Text #11

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2706. Min Act: -0.1700

Data Index: 7750836 (Open Web Text)

Max Activating Token Index: 70

Text #12

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2755. Min Act: -0.1700

Data Index: 7003452 (Open Web Text)

Max Activating Token Index: 568

Text #13

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2718. Min Act: -0.1700

Data Index: 7750835 (Open Web Text)

Max Activating Token Index: 94

Text #14

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2717. Min Act: -0.1700

Data Index: 1670710 (Open Web Text)

Max Activating Token Index: 198

Text #15

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2810. Min Act: -0.1700

Data Index: 2422099 (Open Web Text)

Max Activating Token Index: 743

Text #16

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2720. Min Act: -0.1700

Data Index: 228707 (Open Web Text)

Max Activating Token Index: 777

Text #17

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2864. Min Act: -0.1700

Data Index: 3118195 (Open Web Text)

Max Activating Token Index: 490

Text #18

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2501. Min Act: -0.1700

Data Index: 4774131 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 3.3659. Min Range: -3.3659

Max Act: 3.2561. Min Act: -0.1700

Data Index: 1889270 (Open Web Text)

Max Activating Token Index: 306