Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 317 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4796. Min Act: -0.1700

Data Index: 1227704 (Open Web Text)

Max Activating Token Index: 900

Text #1

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4766. Min Act: -0.1700

Data Index: 7719065 (Open Web Text)

Max Activating Token Index: 932

Text #2

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3786. Min Act: -0.1700

Data Index: 4056141 (Open Web Text)

Max Activating Token Index: 991

Text #3

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4400. Min Act: -0.1700

Data Index: 4452492 (Open Web Text)

Max Activating Token Index: 14

Text #4

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4192. Min Act: -0.1700

Data Index: 3772899 (Open Web Text)

Max Activating Token Index: 260

Text #5

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4115. Min Act: -0.1700

Data Index: 5456498 (Open Web Text)

Max Activating Token Index: 621

Text #6

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.4013. Min Act: -0.1700

Data Index: 5679472 (Open Web Text)

Max Activating Token Index: 35

Text #7

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3980. Min Act: -0.1700

Data Index: 6222909 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3534. Min Act: -0.1700

Data Index: 7429361 (Open Web Text)

Max Activating Token Index: 349

Text #9

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3385. Min Act: -0.1700

Data Index: 6752111 (Open Web Text)

Max Activating Token Index: 837

Text #10

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3209. Min Act: -0.1700

Data Index: 1543058 (Open Web Text)

Max Activating Token Index: 356

Text #11

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3602. Min Act: -0.1700

Data Index: 2002258 (Open Web Text)

Max Activating Token Index: 991

Text #12

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3334. Min Act: -0.1700

Data Index: 4324625 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3442. Min Act: -0.1700

Data Index: 4290248 (Open Web Text)

Max Activating Token Index: 993

Text #14

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3280. Min Act: -0.1700

Data Index: 4181009 (Open Web Text)

Max Activating Token Index: 768

Text #15

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3078. Min Act: -0.1700

Data Index: 1336287 (Open Web Text)

Max Activating Token Index: 886

Text #16

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3146. Min Act: -0.1700

Data Index: 6530277 (Open Web Text)

Max Activating Token Index: 919

Text #17

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3446. Min Act: -0.1700

Data Index: 293374 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.2866. Min Act: -0.1700

Data Index: 5491852 (Open Web Text)

Max Activating Token Index: 342

Text #19

Max Range: 4.4796. Min Range: -4.4796

Max Act: 4.3091. Min Act: -0.1700

Data Index: 5403897 (Open Web Text)

Max Activating Token Index: 492