Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 192 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.9128. Min Act: -0.1700

Data Index: 7253547 (Open Web Text)

Max Activating Token Index: 22

Text #1

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.7877. Min Act: -0.1700

Data Index: 8267647 (Open Web Text)

Max Activating Token Index: 200

Text #2

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6907. Min Act: -0.1700

Data Index: 7387560 (Open Web Text)

Max Activating Token Index: 727

Text #3

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6756. Min Act: -0.1700

Data Index: 2974104 (Open Web Text)

Max Activating Token Index: 416

Text #4

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6971. Min Act: -0.1700

Data Index: 6010323 (Open Web Text)

Max Activating Token Index: 546

Text #5

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6780. Min Act: -0.1700

Data Index: 6333428 (Open Web Text)

Max Activating Token Index: 483

Text #6

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6654. Min Act: -0.1700

Data Index: 1890368 (Open Web Text)

Max Activating Token Index: 379

Text #7

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6796. Min Act: -0.1700

Data Index: 8077345 (Open Web Text)

Max Activating Token Index: 362

Text #8

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6786. Min Act: -0.1700

Data Index: 4761102 (Open Web Text)

Max Activating Token Index: 415

Text #9

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6468. Min Act: -0.1700

Data Index: 6104544 (Open Web Text)

Max Activating Token Index: 282

Text #10

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6072. Min Act: -0.1700

Data Index: 2479897 (Open Web Text)

Max Activating Token Index: 466

Text #11

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.6026. Min Act: -0.1700

Data Index: 1482359 (Open Web Text)

Max Activating Token Index: 129

Text #12

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5771. Min Act: -0.1700

Data Index: 6857914 (Open Web Text)

Max Activating Token Index: 748

Text #13

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5641. Min Act: -0.1700

Data Index: 2482708 (Open Web Text)

Max Activating Token Index: 460

Text #14

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5794. Min Act: -0.1700

Data Index: 972888 (Open Web Text)

Max Activating Token Index: 134

Text #15

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5782. Min Act: -0.1700

Data Index: 2695903 (Open Web Text)

Max Activating Token Index: 436

Text #16

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5675. Min Act: -0.1700

Data Index: 1882058 (Open Web Text)

Max Activating Token Index: 387

Text #17

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5721. Min Act: -0.1700

Data Index: 3748482 (Open Web Text)

Max Activating Token Index: 288

Text #18

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5402. Min Act: -0.1700

Data Index: 3342399 (Open Web Text)

Max Activating Token Index: 39

Text #19

Max Range: 3.9128. Min Range: -3.9128

Max Act: 3.5607. Min Act: -0.1700

Data Index: 3652123 (Open Web Text)

Max Activating Token Index: 959