Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1118 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0114. Min Range: -2.0114

Max Act: 2.0114. Min Act: -0.1700

Data Index: 8354049 (Open Web Text)

Max Activating Token Index: 215

Text #1

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9750. Min Act: -0.1700

Data Index: 159995 (Open Web Text)

Max Activating Token Index: 460

Text #2

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9383. Min Act: -0.1700

Data Index: 3491336 (Open Web Text)

Max Activating Token Index: 296

Text #3

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9314. Min Act: -0.1700

Data Index: 3018277 (Open Web Text)

Max Activating Token Index: 789

Text #4

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9170. Min Act: -0.1700

Data Index: 6298159 (Open Web Text)

Max Activating Token Index: 184

Text #5

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9092. Min Act: -0.1700

Data Index: 2398469 (Open Web Text)

Max Activating Token Index: 189

Text #6

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9057. Min Act: -0.1700

Data Index: 8167685 (Open Web Text)

Max Activating Token Index: 462

Text #7

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9066. Min Act: -0.1700

Data Index: 1776571 (Open Web Text)

Max Activating Token Index: 881

Text #8

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9009. Min Act: -0.1700

Data Index: 1145122 (Open Web Text)

Max Activating Token Index: 670

Text #9

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9018. Min Act: -0.1700

Data Index: 6198259 (Open Web Text)

Max Activating Token Index: 792

Text #10

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8958. Min Act: -0.1700

Data Index: 503706 (Open Web Text)

Max Activating Token Index: 718

Text #11

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8962. Min Act: -0.1700

Data Index: 226238 (Open Web Text)

Max Activating Token Index: 658

Text #12

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8891. Min Act: -0.1700

Data Index: 4333743 (Open Web Text)

Max Activating Token Index: 83

Text #13

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.9029. Min Act: -0.1700

Data Index: 8015911 (Open Web Text)

Max Activating Token Index: 776

Text #14

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8861. Min Act: -0.1700

Data Index: 2732718 (Open Web Text)

Max Activating Token Index: 711

Text #15

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8909. Min Act: -0.1700

Data Index: 226239 (Open Web Text)

Max Activating Token Index: 818

Text #16

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8793. Min Act: -0.1700

Data Index: 6965857 (Open Web Text)

Max Activating Token Index: 877

Text #17

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8852. Min Act: -0.1700

Data Index: 2087082 (Open Web Text)

Max Activating Token Index: 496

Text #18

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8781. Min Act: -0.1700

Data Index: 3181004 (Open Web Text)

Max Activating Token Index: 947

Text #19

Max Range: 2.0114. Min Range: -2.0114

Max Act: 1.8781. Min Act: -0.1700

Data Index: 84285 (Open Web Text)

Max Activating Token Index: 936