Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2991 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.4977. Min Act: -0.0956

Data Index: 4081144 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.3070. Min Act: -0.0000

Data Index: 7782289 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.2951. Min Act: -0.0653

Data Index: 5606813 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.2517. Min Act: -0.1071

Data Index: 5953752 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.2816. Min Act: -0.1043

Data Index: 7088714 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.2605. Min Act: -0.1220

Data Index: 770604 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1946. Min Act: -0.0526

Data Index: 5796848 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1671. Min Act: -0.1473

Data Index: 4551864 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1800. Min Act: -0.1420

Data Index: 601930 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1881. Min Act: -0.0000

Data Index: 6614441 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1523. Min Act: -0.1493

Data Index: 8748698 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1866. Min Act: -0.0725

Data Index: 3432282 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1373. Min Act: -0.0948

Data Index: 1673037 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1333. Min Act: -0.0441

Data Index: 6036241 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1097. Min Act: -0.1181

Data Index: 6090024 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1248. Min Act: -0.0575

Data Index: 2507261 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1415. Min Act: -0.0660

Data Index: 7023935 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1264. Min Act: -0.1151

Data Index: 6418633 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1208. Min Act: -0.1056

Data Index: 651399 (Open Web Text)

Max Activating Token Index: 1

Text #19

Max Range: 2.4977. Min Range: -2.4977

Max Act: 2.1208. Min Act: -0.1506

Data Index: 4827533 (Open Web Text)

Max Activating Token Index: 1