Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1193 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.5191. Min Act: -0.1700

Data Index: 2886803 (Open Web Text)

Max Activating Token Index: 224

Text #1

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.5005. Min Act: -0.1700

Data Index: 2308364 (Open Web Text)

Max Activating Token Index: 986

Text #2

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.5020. Min Act: -0.1700

Data Index: 8652021 (Open Web Text)

Max Activating Token Index: 759

Text #3

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.5191. Min Act: -0.1700

Data Index: 6415207 (Open Web Text)

Max Activating Token Index: 453

Text #4

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.5135. Min Act: -0.1700

Data Index: 2628395 (Open Web Text)

Max Activating Token Index: 209

Text #5

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4802. Min Act: -0.1700

Data Index: 7940516 (Open Web Text)

Max Activating Token Index: 517

Text #6

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4505. Min Act: -0.1700

Data Index: 1190131 (Open Web Text)

Max Activating Token Index: 512

Text #7

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4536. Min Act: -0.1700

Data Index: 2237716 (Open Web Text)

Max Activating Token Index: 638

Text #8

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4342. Min Act: -0.1700

Data Index: 2157445 (Open Web Text)

Max Activating Token Index: 994

Text #9

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4309. Min Act: -0.1700

Data Index: 1118515 (Open Web Text)

Max Activating Token Index: 492

Text #10

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4452. Min Act: -0.1700

Data Index: 1523772 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4216. Min Act: -0.1700

Data Index: 7285320 (Open Web Text)

Max Activating Token Index: 646

Text #12

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4262. Min Act: -0.1700

Data Index: 2597615 (Open Web Text)

Max Activating Token Index: 707

Text #13

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4133. Min Act: -0.1700

Data Index: 2004340 (Open Web Text)

Max Activating Token Index: 348

Text #14

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4077. Min Act: -0.1700

Data Index: 2451337 (Open Web Text)

Max Activating Token Index: 97

Text #15

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4155. Min Act: -0.1700

Data Index: 3088022 (Open Web Text)

Max Activating Token Index: 467

Text #16

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4142. Min Act: -0.1700

Data Index: 3149432 (Open Web Text)

Max Activating Token Index: 432

Text #17

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.3980. Min Act: -0.1700

Data Index: 7591613 (Open Web Text)

Max Activating Token Index: 778

Text #18

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.3991. Min Act: -0.1700

Data Index: 1311308 (Open Web Text)

Max Activating Token Index: 13

Text #19

Max Range: 3.5191. Min Range: -3.5191

Max Act: 3.4102. Min Act: -0.1700

Data Index: 3162179 (Open Web Text)

Max Activating Token Index: 266