Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1368 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.1703. Min Act: -0.1700

Data Index: 6275197 (Open Web Text)

Max Activating Token Index: 160

Text #1

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.2200. Min Act: -0.1700

Data Index: 7177548 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.1696. Min Act: -0.1700

Data Index: 1966701 (Open Web Text)

Max Activating Token Index: 81

Text #3

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.1705. Min Act: -0.1700

Data Index: 1950018 (Open Web Text)

Max Activating Token Index: 550

Text #4

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0755. Min Act: -0.1700

Data Index: 431115 (Open Web Text)

Max Activating Token Index: 52

Text #5

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0625. Min Act: -0.1700

Data Index: 2627486 (Open Web Text)

Max Activating Token Index: 82

Text #6

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0340. Min Act: -0.1700

Data Index: 4515972 (Open Web Text)

Max Activating Token Index: 184

Text #7

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0300. Min Act: -0.1700

Data Index: 2040435 (Open Web Text)

Max Activating Token Index: 368

Text #8

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0178. Min Act: -0.1700

Data Index: 4795102 (Open Web Text)

Max Activating Token Index: 583

Text #9

Max Range: 3.2200. Min Range: -3.2200

Max Act: 2.9942. Min Act: -0.1700

Data Index: 7556328 (Open Web Text)

Max Activating Token Index: 913

Text #10

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0204. Min Act: -0.1700

Data Index: 7770239 (Open Web Text)

Max Activating Token Index: 248

Text #11

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0047. Min Act: -0.1700

Data Index: 2724106 (Open Web Text)

Max Activating Token Index: 320

Text #12

Max Range: 3.2200. Min Range: -3.2200

Max Act: 2.9892. Min Act: -0.1700

Data Index: 6267710 (Open Web Text)

Max Activating Token Index: 200

Text #13

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0158. Min Act: -0.1700

Data Index: 7237111 (Open Web Text)

Max Activating Token Index: 757

Text #14

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0030. Min Act: -0.1700

Data Index: 6267712 (Open Web Text)

Max Activating Token Index: 818

Text #15

Max Range: 3.2200. Min Range: -3.2200

Max Act: 2.9974. Min Act: -0.1700

Data Index: 3969584 (Open Web Text)

Max Activating Token Index: 921

Text #16

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0205. Min Act: -0.1700

Data Index: 8803631 (Open Web Text)

Max Activating Token Index: 908

Text #17

Max Range: 3.2200. Min Range: -3.2200

Max Act: 3.0045. Min Act: -0.1700

Data Index: 731854 (Open Web Text)

Max Activating Token Index: 570

Text #18

Max Range: 3.2200. Min Range: -3.2200

Max Act: 2.9862. Min Act: -0.1700

Data Index: 5031088 (Open Web Text)

Max Activating Token Index: 129

Text #19

Max Range: 3.2200. Min Range: -3.2200

Max Act: 2.9674. Min Act: -0.1700

Data Index: 2265566 (Open Web Text)

Max Activating Token Index: 325