Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 998 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.8952. Min Act: -0.1700

Data Index: 8462254 (Open Web Text)

Max Activating Token Index: 591

Text #1

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.8794. Min Act: -0.1700

Data Index: 6199574 (Open Web Text)

Max Activating Token Index: 399

Text #2

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.7389. Min Act: -0.1700

Data Index: 6240256 (Open Web Text)

Max Activating Token Index: 532

Text #3

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6935. Min Act: -0.1700

Data Index: 5695595 (Open Web Text)

Max Activating Token Index: 508

Text #4

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6642. Min Act: -0.1700

Data Index: 270257 (Open Web Text)

Max Activating Token Index: 769

Text #5

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6084. Min Act: -0.1700

Data Index: 6283307 (Open Web Text)

Max Activating Token Index: 550

Text #6

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6112. Min Act: -0.1700

Data Index: 5570683 (Open Web Text)

Max Activating Token Index: 476

Text #7

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6096. Min Act: -0.1700

Data Index: 7631686 (Open Web Text)

Max Activating Token Index: 683

Text #8

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.6003. Min Act: -0.1700

Data Index: 8457255 (Open Web Text)

Max Activating Token Index: 840

Text #9

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5775. Min Act: -0.1700

Data Index: 1349763 (Open Web Text)

Max Activating Token Index: 613

Text #10

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5918. Min Act: -0.1700

Data Index: 6784498 (Open Web Text)

Max Activating Token Index: 1004

Text #11

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5275. Min Act: -0.1700

Data Index: 6101879 (Open Web Text)

Max Activating Token Index: 734

Text #12

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5708. Min Act: -0.1700

Data Index: 2963638 (Open Web Text)

Max Activating Token Index: 121

Text #13

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5631. Min Act: -0.1700

Data Index: 6660936 (Open Web Text)

Max Activating Token Index: 519

Text #14

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5342. Min Act: -0.1700

Data Index: 7342197 (Open Web Text)

Max Activating Token Index: 778

Text #15

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.5187. Min Act: -0.1700

Data Index: 7650273 (Open Web Text)

Max Activating Token Index: 120

Text #16

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.4918. Min Act: -0.1700

Data Index: 5626269 (Open Web Text)

Max Activating Token Index: 971

Text #17

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.4970. Min Act: -0.1700

Data Index: 3457833 (Open Web Text)

Max Activating Token Index: 668

Text #18

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.4956. Min Act: -0.1700

Data Index: 2264254 (Open Web Text)

Max Activating Token Index: 413

Text #19

Max Range: 3.8952. Min Range: -3.8952

Max Act: 3.4652. Min Act: -0.1700

Data Index: 6454890 (Open Web Text)

Max Activating Token Index: 467