Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1946 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.2888. Min Act: -0.1700

Data Index: 3862940 (Open Web Text)

Max Activating Token Index: 15

Text #1

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.2985. Min Act: -0.1700

Data Index: 135946 (Open Web Text)

Max Activating Token Index: 11

Text #2

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.2946. Min Act: -0.1700

Data Index: 5018235 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.2431. Min Act: -0.1700

Data Index: 4685054 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1883. Min Act: -0.1700

Data Index: 913984 (Open Web Text)

Max Activating Token Index: 24

Text #5

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1576. Min Act: -0.1700

Data Index: 3138369 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1654. Min Act: -0.1700

Data Index: 1989959 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1795. Min Act: -0.1700

Data Index: 3789254 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1795. Min Act: -0.1700

Data Index: 5386133 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1664. Min Act: -0.1700

Data Index: 8369861 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1662. Min Act: -0.1700

Data Index: 7879270 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1637. Min Act: -0.1700

Data Index: 910013 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1681. Min Act: -0.1700

Data Index: 1845414 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1847. Min Act: -0.1700

Data Index: 4780398 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1913. Min Act: -0.1700

Data Index: 6943604 (Open Web Text)

Max Activating Token Index: 34

Text #15

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1833. Min Act: -0.1700

Data Index: 5901040 (Open Web Text)

Max Activating Token Index: 28

Text #16

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1710. Min Act: -0.1700

Data Index: 5519188 (Open Web Text)

Max Activating Token Index: 11

Text #17

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1685. Min Act: -0.1700

Data Index: 4468907 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1313. Min Act: -0.1700

Data Index: 1171931 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 5.2985. Min Range: -5.2985

Max Act: 5.1429. Min Act: -0.1700

Data Index: 5400099 (Open Web Text)

Max Activating Token Index: 10