Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 158 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.3960. Min Act: -0.1700

Data Index: 1650534 (Open Web Text)

Max Activating Token Index: 975

Text #1

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1794. Min Act: -0.1700

Data Index: 3618370 (Open Web Text)

Max Activating Token Index: 838

Text #2

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1579. Min Act: -0.1700

Data Index: 7559156 (Open Web Text)

Max Activating Token Index: 494

Text #3

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1751. Min Act: -0.1700

Data Index: 6161015 (Open Web Text)

Max Activating Token Index: 960

Text #4

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1572. Min Act: -0.1700

Data Index: 8469572 (Open Web Text)

Max Activating Token Index: 482

Text #5

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1668. Min Act: -0.1700

Data Index: 1894475 (Open Web Text)

Max Activating Token Index: 469

Text #6

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1474. Min Act: -0.1700

Data Index: 2767885 (Open Web Text)

Max Activating Token Index: 185

Text #7

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1888. Min Act: -0.1700

Data Index: 2677051 (Open Web Text)

Max Activating Token Index: 15

Text #8

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1154. Min Act: -0.1700

Data Index: 3242256 (Open Web Text)

Max Activating Token Index: 671

Text #9

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1322. Min Act: -0.1700

Data Index: 988914 (Open Web Text)

Max Activating Token Index: 19

Text #10

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.0956. Min Act: -0.1700

Data Index: 201150 (Open Web Text)

Max Activating Token Index: 899

Text #11

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.0982. Min Act: -0.1699

Data Index: 2777337 (Open Web Text)

Max Activating Token Index: 816

Text #12

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1014. Min Act: -0.1700

Data Index: 7329140 (Open Web Text)

Max Activating Token Index: 147

Text #13

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.2371. Min Act: -0.1700

Data Index: 8124453 (Open Web Text)

Max Activating Token Index: 1000

Text #14

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.0653. Min Act: -0.1700

Data Index: 2271515 (Open Web Text)

Max Activating Token Index: 453

Text #15

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1343. Min Act: -0.1700

Data Index: 5537693 (Open Web Text)

Max Activating Token Index: 611

Text #16

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1065. Min Act: -0.1700

Data Index: 5463691 (Open Web Text)

Max Activating Token Index: 426

Text #17

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1026. Min Act: -0.1700

Data Index: 4201884 (Open Web Text)

Max Activating Token Index: 76

Text #18

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.1269. Min Act: -0.1700

Data Index: 4739851 (Open Web Text)

Max Activating Token Index: 331

Text #19

Max Range: 5.3960. Min Range: -5.3960

Max Act: 5.0777. Min Act: -0.1700

Data Index: 3001592 (Open Web Text)

Max Activating Token Index: 64