Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2960 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.3428. Min Act: -0.1700

Data Index: 4248387 (Open Web Text)

Max Activating Token Index: 238

Text #1

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.2972. Min Act: -0.1700

Data Index: 7542198 (Open Web Text)

Max Activating Token Index: 426

Text #2

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.2794. Min Act: -0.1700

Data Index: 4520673 (Open Web Text)

Max Activating Token Index: 189

Text #3

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.3060. Min Act: -0.1700

Data Index: 3954873 (Open Web Text)

Max Activating Token Index: 931

Text #4

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.2331. Min Act: -0.1700

Data Index: 847376 (Open Web Text)

Max Activating Token Index: 451

Text #5

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.2029. Min Act: -0.1700

Data Index: 4520672 (Open Web Text)

Max Activating Token Index: 845

Text #6

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.2399. Min Act: -0.1700

Data Index: 3510740 (Open Web Text)

Max Activating Token Index: 17

Text #7

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1999. Min Act: -0.1700

Data Index: 3063865 (Open Web Text)

Max Activating Token Index: 899

Text #8

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1901. Min Act: -0.1700

Data Index: 7279206 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1684. Min Act: -0.1700

Data Index: 6944552 (Open Web Text)

Max Activating Token Index: 929

Text #10

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1584. Min Act: -0.1700

Data Index: 1528059 (Open Web Text)

Max Activating Token Index: 640

Text #11

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1460. Min Act: -0.1700

Data Index: 7966983 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1625. Min Act: -0.1700

Data Index: 882809 (Open Web Text)

Max Activating Token Index: 206

Text #13

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1562. Min Act: -0.1700

Data Index: 2935518 (Open Web Text)

Max Activating Token Index: 475

Text #14

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1388. Min Act: -0.1700

Data Index: 1494619 (Open Web Text)

Max Activating Token Index: 580

Text #15

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1392. Min Act: -0.1700

Data Index: 6828583 (Open Web Text)

Max Activating Token Index: 296

Text #16

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1549. Min Act: -0.1700

Data Index: 888207 (Open Web Text)

Max Activating Token Index: 997

Text #17

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1230. Min Act: -0.1700

Data Index: 1062998 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1197. Min Act: -0.1700

Data Index: 3733995 (Open Web Text)

Max Activating Token Index: 985

Text #19

Max Range: 3.3428. Min Range: -3.3428

Max Act: 3.1199. Min Act: -0.1700

Data Index: 5414277 (Open Web Text)

Max Activating Token Index: 764