Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2166 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.4796. Min Act: -0.1700

Data Index: 3630431 (Open Web Text)

Max Activating Token Index: 308

Text #1

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.4175. Min Act: -0.1700

Data Index: 2263503 (Open Web Text)

Max Activating Token Index: 659

Text #2

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3869. Min Act: -0.1700

Data Index: 3480405 (Open Web Text)

Max Activating Token Index: 445

Text #3

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3988. Min Act: -0.1700

Data Index: 8476112 (Open Web Text)

Max Activating Token Index: 522

Text #4

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3532. Min Act: -0.1700

Data Index: 5025383 (Open Web Text)

Max Activating Token Index: 30

Text #5

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3290. Min Act: -0.1700

Data Index: 7474667 (Open Web Text)

Max Activating Token Index: 814

Text #6

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3485. Min Act: -0.1700

Data Index: 3511836 (Open Web Text)

Max Activating Token Index: 123

Text #7

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3150. Min Act: -0.1700

Data Index: 7474661 (Open Web Text)

Max Activating Token Index: 855

Text #8

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3207. Min Act: -0.1700

Data Index: 6824304 (Open Web Text)

Max Activating Token Index: 743

Text #9

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3223. Min Act: -0.1700

Data Index: 7527915 (Open Web Text)

Max Activating Token Index: 546

Text #10

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3033. Min Act: -0.1700

Data Index: 4367080 (Open Web Text)

Max Activating Token Index: 401

Text #11

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.3066. Min Act: -0.1700

Data Index: 6291034 (Open Web Text)

Max Activating Token Index: 518

Text #12

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2896. Min Act: -0.1700

Data Index: 5809286 (Open Web Text)

Max Activating Token Index: 620

Text #13

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2903. Min Act: -0.1700

Data Index: 6473637 (Open Web Text)

Max Activating Token Index: 102

Text #14

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2759. Min Act: -0.1700

Data Index: 1117783 (Open Web Text)

Max Activating Token Index: 128

Text #15

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2915. Min Act: -0.1700

Data Index: 61424 (Open Web Text)

Max Activating Token Index: 736

Text #16

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2853. Min Act: -0.1700

Data Index: 1372500 (Open Web Text)

Max Activating Token Index: 418

Text #17

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2661. Min Act: -0.1700

Data Index: 4367082 (Open Web Text)

Max Activating Token Index: 990

Text #18

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2584. Min Act: -0.1700

Data Index: 4367081 (Open Web Text)

Max Activating Token Index: 311

Text #19

Max Range: 3.4796. Min Range: -3.4796

Max Act: 3.2669. Min Act: -0.1700

Data Index: 7237079 (Open Web Text)

Max Activating Token Index: 459