Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2047 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.2002. Min Act: -0.1700

Data Index: 6686881 (Open Web Text)

Max Activating Token Index: 262

Text #1

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1859. Min Act: -0.1700

Data Index: 6280767 (Open Web Text)

Max Activating Token Index: 303

Text #2

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1474. Min Act: -0.1700

Data Index: 5940377 (Open Web Text)

Max Activating Token Index: 763

Text #3

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1687. Min Act: -0.1700

Data Index: 6049919 (Open Web Text)

Max Activating Token Index: 124

Text #4

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1550. Min Act: -0.1700

Data Index: 3673880 (Open Web Text)

Max Activating Token Index: 387

Text #5

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1597. Min Act: -0.1700

Data Index: 7681086 (Open Web Text)

Max Activating Token Index: 837

Text #6

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1148. Min Act: -0.1700

Data Index: 5333650 (Open Web Text)

Max Activating Token Index: 825

Text #7

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1369. Min Act: -0.1700

Data Index: 1627042 (Open Web Text)

Max Activating Token Index: 222

Text #8

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0988. Min Act: -0.1700

Data Index: 2775303 (Open Web Text)

Max Activating Token Index: 893

Text #9

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1085. Min Act: -0.1700

Data Index: 2292060 (Open Web Text)

Max Activating Token Index: 181

Text #10

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.1005. Min Act: -0.1700

Data Index: 3059587 (Open Web Text)

Max Activating Token Index: 546

Text #11

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0849. Min Act: -0.1700

Data Index: 1573210 (Open Web Text)

Max Activating Token Index: 49

Text #12

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0736. Min Act: -0.1700

Data Index: 4617668 (Open Web Text)

Max Activating Token Index: 746

Text #13

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0714. Min Act: -0.1700

Data Index: 4341976 (Open Web Text)

Max Activating Token Index: 85

Text #14

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0757. Min Act: -0.1700

Data Index: 6020465 (Open Web Text)

Max Activating Token Index: 512

Text #15

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0350. Min Act: -0.1700

Data Index: 6015823 (Open Web Text)

Max Activating Token Index: 513

Text #16

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0624. Min Act: -0.1700

Data Index: 8427564 (Open Web Text)

Max Activating Token Index: 622

Text #17

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0663. Min Act: -0.1700

Data Index: 3666094 (Open Web Text)

Max Activating Token Index: 310

Text #18

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0702. Min Act: -0.1700

Data Index: 306510 (Open Web Text)

Max Activating Token Index: 555

Text #19

Max Range: 4.2002. Min Range: -4.2002

Max Act: 4.0501. Min Act: -0.1700

Data Index: 1441627 (Open Web Text)

Max Activating Token Index: 251