Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1277 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.3228. Min Act: -0.1700

Data Index: 4734892 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.3143. Min Act: -0.1700

Data Index: 2645036 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.3435. Min Act: -0.1700

Data Index: 5613941 (Open Web Text)

Max Activating Token Index: 940

Text #3

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2870. Min Act: -0.1700

Data Index: 8075005 (Open Web Text)

Max Activating Token Index: 842

Text #4

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2701. Min Act: -0.1700

Data Index: 6772701 (Open Web Text)

Max Activating Token Index: 922

Text #5

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2604. Min Act: -0.1700

Data Index: 5992706 (Open Web Text)

Max Activating Token Index: 32

Text #6

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2488. Min Act: -0.1700

Data Index: 4779036 (Open Web Text)

Max Activating Token Index: 835

Text #7

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2401. Min Act: -0.1700

Data Index: 5200112 (Open Web Text)

Max Activating Token Index: 556

Text #8

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2433. Min Act: -0.1700

Data Index: 7390405 (Open Web Text)

Max Activating Token Index: 828

Text #9

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2465. Min Act: -0.1700

Data Index: 1733754 (Open Web Text)

Max Activating Token Index: 32

Text #10

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2394. Min Act: -0.1700

Data Index: 4303005 (Open Web Text)

Max Activating Token Index: 11

Text #11

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2426. Min Act: -0.1700

Data Index: 2657144 (Open Web Text)

Max Activating Token Index: 57

Text #12

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2338. Min Act: -0.1700

Data Index: 7236423 (Open Web Text)

Max Activating Token Index: 98

Text #13

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2228. Min Act: -0.1700

Data Index: 405322 (Open Web Text)

Max Activating Token Index: 231

Text #14

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2221. Min Act: -0.1700

Data Index: 8196731 (Open Web Text)

Max Activating Token Index: 780

Text #15

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2336. Min Act: -0.1700

Data Index: 3891853 (Open Web Text)

Max Activating Token Index: 1021

Text #16

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2347. Min Act: -0.1700

Data Index: 5306974 (Open Web Text)

Max Activating Token Index: 36

Text #17

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2094. Min Act: -0.1700

Data Index: 1102032 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2146. Min Act: -0.1700

Data Index: 4295674 (Open Web Text)

Max Activating Token Index: 450

Text #19

Max Range: 3.3435. Min Range: -3.3435

Max Act: 3.2233. Min Act: -0.1700

Data Index: 6245477 (Open Web Text)

Max Activating Token Index: 66