Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1558 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.5333. Min Act: -0.1700

Data Index: 4153462 (Open Web Text)

Max Activating Token Index: 824

Text #1

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.5063. Min Act: -0.1700

Data Index: 3508942 (Open Web Text)

Max Activating Token Index: 298

Text #2

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.5027. Min Act: -0.1700

Data Index: 233800 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4952. Min Act: -0.1700

Data Index: 3910045 (Open Web Text)

Max Activating Token Index: 753

Text #4

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4676. Min Act: -0.1700

Data Index: 6267396 (Open Web Text)

Max Activating Token Index: 387

Text #5

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4963. Min Act: -0.1700

Data Index: 7399674 (Open Web Text)

Max Activating Token Index: 757

Text #6

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4784. Min Act: -0.1700

Data Index: 4691890 (Open Web Text)

Max Activating Token Index: 280

Text #7

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4840. Min Act: -0.1700

Data Index: 8706307 (Open Web Text)

Max Activating Token Index: 522

Text #8

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4262. Min Act: -0.1700

Data Index: 4042350 (Open Web Text)

Max Activating Token Index: 869

Text #9

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4651. Min Act: -0.1700

Data Index: 2719313 (Open Web Text)

Max Activating Token Index: 681

Text #10

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4099. Min Act: -0.1700

Data Index: 8649843 (Open Web Text)

Max Activating Token Index: 793

Text #11

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4298. Min Act: -0.1700

Data Index: 5774724 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4191. Min Act: -0.1700

Data Index: 4634196 (Open Web Text)

Max Activating Token Index: 348

Text #13

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4089. Min Act: -0.1700

Data Index: 3393715 (Open Web Text)

Max Activating Token Index: 216

Text #14

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4138. Min Act: -0.1700

Data Index: 2797848 (Open Web Text)

Max Activating Token Index: 422

Text #15

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4347. Min Act: -0.1700

Data Index: 4842252 (Open Web Text)

Max Activating Token Index: 907

Text #16

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4441. Min Act: -0.1700

Data Index: 1641527 (Open Web Text)

Max Activating Token Index: 542

Text #17

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4235. Min Act: -0.1700

Data Index: 1638052 (Open Web Text)

Max Activating Token Index: 362

Text #18

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4101. Min Act: -0.1700

Data Index: 1179030 (Open Web Text)

Max Activating Token Index: 882

Text #19

Max Range: 3.5333. Min Range: -3.5333

Max Act: 3.4215. Min Act: -0.1700

Data Index: 1827233 (Open Web Text)

Max Activating Token Index: 439