Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2782 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0463. Min Range: -6.0463

Max Act: 6.0463. Min Act: -0.1700

Data Index: 1826476 (Open Web Text)

Max Activating Token Index: 482

Text #1

Max Range: 6.0463. Min Range: -6.0463

Max Act: 6.0339. Min Act: -0.1700

Data Index: 4672068 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.9497. Min Act: -0.1700

Data Index: 3294303 (Open Web Text)

Max Activating Token Index: 14

Text #3

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.9619. Min Act: -0.1700

Data Index: 445492 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.9516. Min Act: -0.1700

Data Index: 4260720 (Open Web Text)

Max Activating Token Index: 318

Text #5

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8983. Min Act: -0.1700

Data Index: 7843792 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8938. Min Act: -0.1700

Data Index: 7106928 (Open Web Text)

Max Activating Token Index: 45

Text #7

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8966. Min Act: -0.1700

Data Index: 113437 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.9072. Min Act: -0.1700

Data Index: 2193366 (Open Web Text)

Max Activating Token Index: 393

Text #9

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8939. Min Act: -0.1700

Data Index: 8809711 (Open Web Text)

Max Activating Token Index: 60

Text #10

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8970. Min Act: -0.1700

Data Index: 8638147 (Open Web Text)

Max Activating Token Index: 11

Text #11

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8980. Min Act: -0.1700

Data Index: 7559394 (Open Web Text)

Max Activating Token Index: 271

Text #12

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8871. Min Act: -0.1700

Data Index: 2201239 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8591. Min Act: -0.1700

Data Index: 2532302 (Open Web Text)

Max Activating Token Index: 417

Text #14

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8978. Min Act: -0.1700

Data Index: 2693869 (Open Web Text)

Max Activating Token Index: 331

Text #15

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8688. Min Act: -0.1700

Data Index: 2751480 (Open Web Text)

Max Activating Token Index: 440

Text #16

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8659. Min Act: -0.1700

Data Index: 4793717 (Open Web Text)

Max Activating Token Index: 64

Text #17

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8721. Min Act: -0.1700

Data Index: 5550230 (Open Web Text)

Max Activating Token Index: 341

Text #18

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8691. Min Act: -0.1700

Data Index: 485625 (Open Web Text)

Max Activating Token Index: 17

Text #19

Max Range: 6.0463. Min Range: -6.0463

Max Act: 5.8814. Min Act: -0.1700

Data Index: 6502991 (Open Web Text)

Max Activating Token Index: 367