Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2394 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.8427. Min Act: -0.1700

Data Index: 1690098 (Open Web Text)

Max Activating Token Index: 647

Text #1

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.8049. Min Act: -0.1700

Data Index: 8126685 (Open Web Text)

Max Activating Token Index: 400

Text #2

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.7115. Min Act: -0.1700

Data Index: 8674007 (Open Web Text)

Max Activating Token Index: 550

Text #3

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.7341. Min Act: -0.1700

Data Index: 2089465 (Open Web Text)

Max Activating Token Index: 494

Text #4

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.7039. Min Act: -0.1700

Data Index: 8487917 (Open Web Text)

Max Activating Token Index: 216

Text #5

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.7311. Min Act: -0.1700

Data Index: 1552889 (Open Web Text)

Max Activating Token Index: 950

Text #6

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6160. Min Act: -0.1700

Data Index: 7184605 (Open Web Text)

Max Activating Token Index: 444

Text #7

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6556. Min Act: -0.1700

Data Index: 5837217 (Open Web Text)

Max Activating Token Index: 21

Text #8

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6537. Min Act: -0.1700

Data Index: 2853664 (Open Web Text)

Max Activating Token Index: 75

Text #9

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6877. Min Act: -0.1700

Data Index: 6547560 (Open Web Text)

Max Activating Token Index: 750

Text #10

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6388. Min Act: -0.1700

Data Index: 2470485 (Open Web Text)

Max Activating Token Index: 349

Text #11

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6299. Min Act: -0.1700

Data Index: 3314255 (Open Web Text)

Max Activating Token Index: 927

Text #12

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6192. Min Act: -0.1700

Data Index: 3726760 (Open Web Text)

Max Activating Token Index: 684

Text #13

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6729. Min Act: -0.1700

Data Index: 5313629 (Open Web Text)

Max Activating Token Index: 301

Text #14

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6286. Min Act: -0.1700

Data Index: 5967356 (Open Web Text)

Max Activating Token Index: 332

Text #15

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6036. Min Act: -0.1700

Data Index: 1360567 (Open Web Text)

Max Activating Token Index: 417

Text #16

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.7078. Min Act: -0.1700

Data Index: 5347907 (Open Web Text)

Max Activating Token Index: 399

Text #17

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.5749. Min Act: -0.1700

Data Index: 526513 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6010. Min Act: -0.1700

Data Index: 7488895 (Open Web Text)

Max Activating Token Index: 150

Text #19

Max Range: 4.8427. Min Range: -4.8427

Max Act: 4.6378. Min Act: -0.1700

Data Index: 4136356 (Open Web Text)

Max Activating Token Index: 279