Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1878 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6815. Min Act: -0.1700

Data Index: 2984994 (Open Web Text)

Max Activating Token Index: 64

Text #1

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6465. Min Act: -0.1700

Data Index: 3734677 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6510. Min Act: -0.1700

Data Index: 4729712 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6639. Min Act: -0.1700

Data Index: 599674 (Open Web Text)

Max Activating Token Index: 658

Text #4

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6528. Min Act: -0.1700

Data Index: 2063751 (Open Web Text)

Max Activating Token Index: 508

Text #5

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6573. Min Act: -0.1700

Data Index: 7226011 (Open Web Text)

Max Activating Token Index: 46

Text #6

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6454. Min Act: -0.1700

Data Index: 4943601 (Open Web Text)

Max Activating Token Index: 38

Text #7

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6432. Min Act: -0.1700

Data Index: 200315 (Open Web Text)

Max Activating Token Index: 443

Text #8

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6316. Min Act: -0.1700

Data Index: 6904474 (Open Web Text)

Max Activating Token Index: 530

Text #9

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6400. Min Act: -0.1700

Data Index: 3442459 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6435. Min Act: -0.1700

Data Index: 7834697 (Open Web Text)

Max Activating Token Index: 734

Text #11

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6253. Min Act: -0.1700

Data Index: 2289341 (Open Web Text)

Max Activating Token Index: 15

Text #12

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6128. Min Act: -0.1700

Data Index: 494268 (Open Web Text)

Max Activating Token Index: 23

Text #13

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6250. Min Act: -0.1700

Data Index: 4051230 (Open Web Text)

Max Activating Token Index: 626

Text #14

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6363. Min Act: -0.1700

Data Index: 3487901 (Open Web Text)

Max Activating Token Index: 49

Text #15

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6257. Min Act: -0.1700

Data Index: 8690347 (Open Web Text)

Max Activating Token Index: 556

Text #16

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6283. Min Act: -0.1700

Data Index: 6480109 (Open Web Text)

Max Activating Token Index: 788

Text #17

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6163. Min Act: -0.1700

Data Index: 885781 (Open Web Text)

Max Activating Token Index: 460

Text #18

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6239. Min Act: -0.1700

Data Index: 3005563 (Open Web Text)

Max Activating Token Index: 614

Text #19

Max Range: 1.6815. Min Range: -1.6815

Max Act: 1.6051. Min Act: -0.1700

Data Index: 6662007 (Open Web Text)

Max Activating Token Index: 40