Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2568 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.6127. Min Act: -0.1700

Data Index: 425398 (Open Web Text)

Max Activating Token Index: 571

Text #1

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.5601. Min Act: -0.1700

Data Index: 8600491 (Open Web Text)

Max Activating Token Index: 694

Text #2

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.5363. Min Act: -0.1700

Data Index: 2809521 (Open Web Text)

Max Activating Token Index: 131

Text #3

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.5220. Min Act: -0.1700

Data Index: 1661638 (Open Web Text)

Max Activating Token Index: 510

Text #4

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.5416. Min Act: -0.1700

Data Index: 8237591 (Open Web Text)

Max Activating Token Index: 990

Text #5

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4949. Min Act: -0.1700

Data Index: 8246492 (Open Web Text)

Max Activating Token Index: 743

Text #6

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4791. Min Act: -0.1700

Data Index: 5967809 (Open Web Text)

Max Activating Token Index: 872

Text #7

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4381. Min Act: -0.1700

Data Index: 5225760 (Open Web Text)

Max Activating Token Index: 882

Text #8

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4642. Min Act: -0.1700

Data Index: 4355906 (Open Web Text)

Max Activating Token Index: 193

Text #9

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4955. Min Act: -0.1700

Data Index: 7958594 (Open Web Text)

Max Activating Token Index: 969

Text #10

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4583. Min Act: -0.1700

Data Index: 2551811 (Open Web Text)

Max Activating Token Index: 260

Text #11

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4491. Min Act: -0.1700

Data Index: 7609899 (Open Web Text)

Max Activating Token Index: 796

Text #12

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4473. Min Act: -0.1700

Data Index: 7762383 (Open Web Text)

Max Activating Token Index: 333

Text #13

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4630. Min Act: -0.1700

Data Index: 2092365 (Open Web Text)

Max Activating Token Index: 964

Text #14

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4571. Min Act: -0.1700

Data Index: 4787888 (Open Web Text)

Max Activating Token Index: 859

Text #15

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.3961. Min Act: -0.1700

Data Index: 7414502 (Open Web Text)

Max Activating Token Index: 557

Text #16

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.3652. Min Act: -0.1700

Data Index: 4184281 (Open Web Text)

Max Activating Token Index: 459

Text #17

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.3725. Min Act: -0.1700

Data Index: 3696404 (Open Web Text)

Max Activating Token Index: 838

Text #18

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4008. Min Act: -0.1700

Data Index: 7288425 (Open Web Text)

Max Activating Token Index: 842

Text #19

Max Range: 3.6127. Min Range: -3.6127

Max Act: 3.4002. Min Act: -0.1700

Data Index: 3720295 (Open Web Text)

Max Activating Token Index: 941