Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1814 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.1550. Min Act: -0.1700

Data Index: 3028531 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.1066. Min Act: -0.1700

Data Index: 7912912 (Open Web Text)

Max Activating Token Index: 810

Text #2

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0820. Min Act: -0.1700

Data Index: 835204 (Open Web Text)

Max Activating Token Index: 102

Text #3

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0821. Min Act: -0.1700

Data Index: 904156 (Open Web Text)

Max Activating Token Index: 266

Text #4

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0878. Min Act: -0.1700

Data Index: 1707220 (Open Web Text)

Max Activating Token Index: 510

Text #5

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0617. Min Act: -0.1700

Data Index: 1781780 (Open Web Text)

Max Activating Token Index: 257

Text #6

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0513. Min Act: -0.1700

Data Index: 3512570 (Open Web Text)

Max Activating Token Index: 105

Text #7

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0799. Min Act: -0.1700

Data Index: 7810908 (Open Web Text)

Max Activating Token Index: 949

Text #8

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0124. Min Act: -0.1700

Data Index: 5733172 (Open Web Text)

Max Activating Token Index: 114

Text #9

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0460. Min Act: -0.1700

Data Index: 4551505 (Open Web Text)

Max Activating Token Index: 961

Text #10

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0789. Min Act: -0.1700

Data Index: 4248619 (Open Web Text)

Max Activating Token Index: 925

Text #11

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0249. Min Act: -0.1700

Data Index: 3118631 (Open Web Text)

Max Activating Token Index: 106

Text #12

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0351. Min Act: -0.1700

Data Index: 945543 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0776. Min Act: -0.1700

Data Index: 7725599 (Open Web Text)

Max Activating Token Index: 918

Text #14

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0134. Min Act: -0.1700

Data Index: 6908083 (Open Web Text)

Max Activating Token Index: 294

Text #15

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0699. Min Act: -0.1700

Data Index: 5556955 (Open Web Text)

Max Activating Token Index: 207

Text #16

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0052. Min Act: -0.1700

Data Index: 3877024 (Open Web Text)

Max Activating Token Index: 383

Text #17

Max Range: 4.1550. Min Range: -4.1550

Max Act: 3.9843. Min Act: -0.1700

Data Index: 1636321 (Open Web Text)

Max Activating Token Index: 493

Text #18

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0140. Min Act: -0.1700

Data Index: 12521 (Open Web Text)

Max Activating Token Index: 65

Text #19

Max Range: 4.1550. Min Range: -4.1550

Max Act: 4.0272. Min Act: -0.1700

Data Index: 5331016 (Open Web Text)

Max Activating Token Index: 659