Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2460 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.4737. Min Act: -0.1700

Data Index: 5584917 (Open Web Text)

Max Activating Token Index: 108

Text #1

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.4628. Min Act: -0.1670

Data Index: 5640265 (Open Web Text)

Max Activating Token Index: 120

Text #2

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.4290. Min Act: -0.1700

Data Index: 5640287 (Open Web Text)

Max Activating Token Index: 68

Text #3

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.4328. Min Act: -0.1700

Data Index: 578681 (Open Web Text)

Max Activating Token Index: 59

Text #4

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.4228. Min Act: -0.1695

Data Index: 5640270 (Open Web Text)

Max Activating Token Index: 65

Text #5

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3956. Min Act: -0.1492

Data Index: 153819 (Open Web Text)

Max Activating Token Index: 769

Text #6

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3897. Min Act: -0.1681

Data Index: 6097257 (Open Web Text)

Max Activating Token Index: 37

Text #7

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3838. Min Act: -0.1700

Data Index: 4655079 (Open Web Text)

Max Activating Token Index: 414

Text #8

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3825. Min Act: -0.1492

Data Index: 4279729 (Open Web Text)

Max Activating Token Index: 881

Text #9

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3815. Min Act: -0.1700

Data Index: 4187036 (Open Web Text)

Max Activating Token Index: 953

Text #10

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3845. Min Act: -0.1700

Data Index: 8514568 (Open Web Text)

Max Activating Token Index: 838

Text #11

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3893. Min Act: -0.1700

Data Index: 4938186 (Open Web Text)

Max Activating Token Index: 203

Text #12

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3756. Min Act: -0.1492

Data Index: 5951806 (Open Web Text)

Max Activating Token Index: 558

Text #13

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3781. Min Act: -0.1700

Data Index: 1458395 (Open Web Text)

Max Activating Token Index: 857

Text #14

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3694. Min Act: -0.1700

Data Index: 3405432 (Open Web Text)

Max Activating Token Index: 107

Text #15

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3792. Min Act: -0.1700

Data Index: 3994449 (Open Web Text)

Max Activating Token Index: 1003

Text #16

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3760. Min Act: -0.1700

Data Index: 6275918 (Open Web Text)

Max Activating Token Index: 144

Text #17

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3705. Min Act: -0.1699

Data Index: 1790454 (Open Web Text)

Max Activating Token Index: 502

Text #18

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3731. Min Act: -0.1492

Data Index: 6256402 (Open Web Text)

Max Activating Token Index: 173

Text #19

Max Range: 1.4737. Min Range: -1.4737

Max Act: 1.3685. Min Act: -0.1492

Data Index: 3762148 (Open Web Text)

Max Activating Token Index: 61