Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2427 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.7702. Min Act: -0.1700

Data Index: 2456313 (Open Web Text)

Max Activating Token Index: 629

Text #1

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.7321. Min Act: -0.1700

Data Index: 2957010 (Open Web Text)

Max Activating Token Index: 880

Text #2

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.7115. Min Act: -0.1700

Data Index: 4028365 (Open Web Text)

Max Activating Token Index: 418

Text #3

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.7069. Min Act: -0.1700

Data Index: 2456314 (Open Web Text)

Max Activating Token Index: 523

Text #4

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6786. Min Act: -0.1700

Data Index: 4822493 (Open Web Text)

Max Activating Token Index: 376

Text #5

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6986. Min Act: -0.1700

Data Index: 1508102 (Open Web Text)

Max Activating Token Index: 859

Text #6

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6486. Min Act: -0.1700

Data Index: 3252577 (Open Web Text)

Max Activating Token Index: 494

Text #7

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6461. Min Act: -0.1700

Data Index: 7036969 (Open Web Text)

Max Activating Token Index: 692

Text #8

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6280. Min Act: -0.1700

Data Index: 4391513 (Open Web Text)

Max Activating Token Index: 257

Text #9

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6072. Min Act: -0.1700

Data Index: 1263852 (Open Web Text)

Max Activating Token Index: 549

Text #10

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6089. Min Act: -0.1700

Data Index: 2456311 (Open Web Text)

Max Activating Token Index: 616

Text #11

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6195. Min Act: -0.1700

Data Index: 7925686 (Open Web Text)

Max Activating Token Index: 777

Text #12

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6187. Min Act: -0.1700

Data Index: 1598351 (Open Web Text)

Max Activating Token Index: 132

Text #13

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.5866. Min Act: -0.1700

Data Index: 336747 (Open Web Text)

Max Activating Token Index: 432

Text #14

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6078. Min Act: -0.1700

Data Index: 8135708 (Open Web Text)

Max Activating Token Index: 822

Text #15

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.5888. Min Act: -0.1700

Data Index: 65067 (Open Web Text)

Max Activating Token Index: 273

Text #16

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6050. Min Act: -0.1700

Data Index: 6110580 (Open Web Text)

Max Activating Token Index: 236

Text #17

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.6225. Min Act: -0.1700

Data Index: 4885997 (Open Web Text)

Max Activating Token Index: 953

Text #18

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.5846. Min Act: -0.1700

Data Index: 4483332 (Open Web Text)

Max Activating Token Index: 465

Text #19

Max Range: 3.7702. Min Range: -3.7702

Max Act: 3.5926. Min Act: -0.1700

Data Index: 4531019 (Open Web Text)

Max Activating Token Index: 320